Corner point sorting method, device and electronic device

Abstract

The embodiments of the present application disclose a corner point sorting method, device, and electronic device. The method includes: obtaining a text area to be processed, the text area including multiple corner points; obtaining a reference sorting of the multiple corner points based on the angle of the text area, the angle representing the positional relationship between the text area and the edge of the image to which the text area belongs; obtaining the direction of the text area, the direction representing the direction vector corresponding to the content in the text area; and obtaining a target sorting of the multiple corner points based on the direction and the reference sorting. Through the above method, a reference sorting of the multiple corner points can be obtained based on the angle representing the positional relationship between the text area and the edge of the image to which the text area belongs, and then a target sorting of the multiple corner points of the text area can be obtained based on the direction of the direction vector corresponding to the content in the text area and the reference sorting, thereby improving the efficiency of downstream tasks.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a corner point sorting method, device, and electronic device. Background Art

[0002] With the widespread adoption of electronic devices like computers and mobile phones, paperless storage—transcription of text from traditional printed materials into computers—is becoming a trend. Against this backdrop, OCR (Optical Character Recognition) technology has been extensively researched, and with the maturity of deep learning technology, OCR algorithms have been significantly improved.

[0003] In related approaches, the steps of an OCR algorithm can be divided into detection and recognition. Detection locates the area where text is located, while recognition identifies the text within that area. However, these approaches fail to consider downstream tasks, resulting in low efficiency. Summary of the Invention

[0004] In view of the above problems, the present application proposes a corner point sorting method, device and electronic device to improve the above problems.

[0005] In a first aspect, the present application provides a corner point sorting method, the method comprising: obtaining a text area to be processed, the text area comprising a plurality of corner points; obtaining a reference sorting of the plurality of corner points based on an angle of the text area, the angle representing a positional relationship between the text area and an edge of an image to which the text area belongs; obtaining a direction of the text area, the direction representing a direction vector corresponding to content in the text area; and obtaining a target sorting of the plurality of corner points based on the direction and the reference sorting.

[0006] In a second aspect, the present application provides a corner point sorting device, which includes: a text area acquisition unit, used to acquire a text area, wherein the text area includes multiple corner points; a reference sorting acquisition unit, used to obtain a reference sorting of the multiple corner points based on the angle of the text area, wherein the angle represents the positional relationship between the text area and the edge of the image to which the text area belongs; a direction acquisition unit, used to acquire the direction of the text area, wherein the direction represents the direction vector corresponding to the content in the text area; and a target sorting acquisition unit, used to obtain a target sorting of the multiple corner points based on the direction and the reference sorting.

[0007] In a third aspect, the present application provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0008] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores program code, wherein the above method is executed when the program code is run.

[0009] The present application provides a corner point sorting method, device, electronic device, and storage medium. After obtaining a text region to be processed that includes multiple corner points, a reference sorting of the multiple corner points is obtained based on an angle representing the positional relationship between the text region and the edge of the image to which the text region belongs. The direction of a direction vector corresponding to the content in the text region is obtained, and a target sorting of the multiple corner points is obtained based on the direction and the reference sorting. Through the above-mentioned method, a reference sorting of the multiple corner points can be obtained based on the angle representing the positional relationship between the text region and the edge of the image to which the text region belongs. Then, a target sorting of the multiple corner points in the text region can be obtained based on the direction of the direction vector corresponding to the content in the text region and the reference sorting. This allows downstream tasks to be performed directly based on the target sorting of the multiple corner points, thereby improving the efficiency of downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A flowchart of a corner point sorting method proposed in an embodiment of the present application is shown;

[0012] Figure 2 A schematic diagram showing a text area proposed in this application;

[0013] Figure 3 A schematic diagram showing a text box angle proposed in this application is shown;

[0014] Figure 4 Shows this application Figure 1 A flowchart of an embodiment of S130;

[0015] Figure 5 A schematic diagram showing a direction vector of a text area proposed in this application is shown;

[0016] Figure 6 A schematic diagram showing the direction of a text area proposed in this application;

[0017] Figure 7A flowchart of a corner point sorting method proposed in another embodiment of the present application is shown;

[0018] Figure 8 A schematic diagram of a sorting vector and a midpoint vector of a text region proposed in this application is shown;

[0019] Figure 9 Shows this application Figure 7 A flowchart of an embodiment of S280;

[0020] Figure 10 A schematic diagram of a first reference vector and a second reference vector proposed in this application is shown;

[0021] Figure 11 A structural block diagram of a corner point sorting device proposed in an embodiment of the present application is shown;

[0022] Figure 12 Shows a structural block diagram of an electronic device proposed in this application;

[0023] Figure 13 It is a storage unit for storing or carrying program codes for implementing the corner point sorting method according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0025] With the widespread adoption of electronic devices and the development of deep learning technology, OCR technology has been extensively studied. In related methods, the steps of OCR algorithms can be divided into detection and recognition. Detection refers to locating the area where text is located, while recognition identifies the text within that area.

[0026] However, the inventors discovered in their related research that the related methods did not take downstream tasks into consideration, resulting in low efficiency of downstream tasks. For example, when a mobile phone opens a picture, the OCR algorithm can recognize the text and text coordinates in it, and the user can swipe a number of text areas with their fingers to perform subsequent operations such as copying, searching, and translating. In order to ensure user experience, certain feedback can be given to the user during the swiping process, such as rendering the selected text area with a blue background. In order to simplify development work, the rendering module requires that the corner points of the text box must be ordered. In the related methods, the corner points output by the OCR algorithm are not ordered, which makes the downstream rendering tasks inefficient.

[0027] Therefore, the inventors proposed a corner point sorting method, device, and electronic device in the present application. After obtaining a text region to be processed that includes multiple corner points, a reference sorting of the multiple corner points is obtained based on an angle representing the positional relationship between the text region and the edge of the image to which the text region belongs. The direction of the direction vector corresponding to the content in the text region is obtained, and a target sorting of the multiple corner points is obtained based on the direction and the reference sorting. Through the above method, a reference sorting of the multiple corner points can be obtained based on the angle representing the positional relationship between the text region and the edge of the image to which the text region belongs. Then, a target sorting of the multiple corner points in the text region can be obtained based on the direction of the direction vector corresponding to the content in the text region and the reference sorting. This allows downstream tasks to be performed directly based on the target sorting of the multiple corner points, thereby improving the efficiency of downstream tasks.

[0028] The embodiments of this application will be described below with reference to the accompanying drawings.

[0029] See also Figure 1 , the present application provides a corner point sorting method, the method comprising:

[0030] S110: Acquire a text region to be processed, where the text region includes a plurality of corner points.

[0031] The text area may refer to a rectangular frame containing content in the image, and the content in the rectangular frame may be text. A corner point may refer to the intersection of two adjacent sides of the text area. For example, Figure 2 As shown, the text area may be a rectangular frame containing "ABCDEFG", and P1, P2, P3, and P4 may be multiple corner points of the text area.

[0032] As a method, the text area to be processed and multiple corner points corresponding to the text area can be obtained based on a detection model (such as a convolutional neural network, etc.).

[0033] Optionally, if the obtained text area is non-rectangular, you can use the minAreaRect function in OpenCV to convert the text area into a rectangle.

[0034] S120: Obtaining a reference ordering of the plurality of corner points based on an angle of the text region, wherein the angle represents a positional relationship between the text region and an edge of an image to which the text region belongs.

[0035] The positional relationship may be that the text region is parallel or non-parallel to the edge of the image to which the text region belongs. The reference sorting may refer to a preliminary sorting result of multiple corner points based on the angle of the text region.

[0036] As one approach, the angle of the text region may be determined based on the corner point coordinates corresponding to each of the multiple corner points.

[0037] Optionally, the corner point coordinates corresponding to multiple corner points can be obtained. The corner point coordinates may include horizontal coordinates and vertical coordinates. If the corner point coordinates corresponding to multiple corner points have the same horizontal coordinate, it is determined that the angle of the text area represents that the text area is parallel to the edge of the image to which the text area belongs; if the horizontal coordinates corresponding to multiple corner points are different, it is determined that the angle of the text area represents that the text area is not parallel to the edge of the image to which the text area belongs.

[0038] For example, Figure 3 As shown in the middle left figure, the coordinates of the corner points P1, P2, P3, and P4 corresponding to the text area are (x p11 ,y p11 )、(x p21 ,y p21 )、(x p31 ,y p31 )、(x p41 ,y p41 ), where the horizontal coordinates of points P1 and P4 are the same, and the horizontal coordinates of points P2 and P3 are the same, then the angle of the text area indicates that the text area is parallel to the edge of the image to which the text area belongs. Figure 3 As shown in the middle right figure, the coordinates of the corner points P1, P2, P3, and P4 corresponding to the text area are (x p12 ,y p12 )、(x p22 ,y p22 )、(x p32 ,y p32 )、(x p42 ,y p42 ), where the horizontal coordinates of all corner points are different, then the angle of the text region represents that the text region is not parallel to the edge of the image to which the text region belongs.

[0039] Optionally, in an embodiment of the present application, the angle representation of the text area can also be determined based on whether the text area is parallel to the edge of the image to which the text area belongs based on whether the corner point coordinates corresponding to multiple corner points have the same vertical coordinate.

[0040] In an embodiment of the present application, after the angle of the text area is obtained, a reference sorting can be obtained in a manner corresponding to the angle.

[0041] Among them, as a method, if the angle representing the text area is parallel to the edge of the image to which the text area belongs, the horizontal coordinate and the vertical coordinate of each corner point can be added to obtain the sum of the coordinates corresponding to multiple corner points; based on the sum of multiple coordinates, the first reference corner point is determined, and the first reference corner point is the corner point ranked first in the reference sorting; based on the first reference corner point, the reference arrangement order of the remaining corner points in the multiple corner points is determined; based on the first reference corner point and the reference arrangement order, the reference sorting is obtained.

[0042] Optionally, the corner point corresponding to the minimum value among multiple coordinate sums can be used as the first reference corner point.

[0043] Exemplarily, as Figure 3 shown in the left figure of 3, the coordinate sums of P1, P2, P3, and P4 are S1, S2, S3, and S4 respectively. Since S1 < S4 < S2 < S3, the first reference can be P1.

[0044] Optionally, based on the distances of the remaining corner points among multiple corner points from the first reference corner point in the clockwise direction, the reference arrangement order of the remaining corner points among multiple corner points can be determined. Among them, the closer the remaining corner point is to the first reference corner point in the clockwise direction, the more forward the reference arrangement order.

[0045] Exemplarily, as shown in the left figure of 3, when the first reference corner point is P1, the reference arrangement order can be: P2, P3, P4.

[0046] Optionally, the first reference corner point can be ranked first in the reference sorting, and then the remaining corner points can be ranked behind the first reference corner point according to the reference arrangement order to obtain the reference sorting.

[0047] Exemplarily, as shown in the left figure of 3, when the first reference corner point is P1 and the reference arrangement order is: P2, P3, P4, the reference sorting can be: P1, P2, P3, P4.

[0048] As another method, if the angle represents that the text region is not parallel to the side of the image to which the text region belongs, the first reference corner point can be determined based on the abscissas corresponding to multiple corner points respectively, and the first reference corner point is the corner point ranked first in the reference sorting; obtain the slopes of the lines corresponding to the remaining corner points among multiple corner points respectively, and the line can be formed by connecting the remaining corner points among multiple corner points with the first reference corner point respectively; determine the reference arrangement order of the remaining corner points among multiple corner points based on the slopes; obtain the reference sorting based on the first reference corner point and the reference arrangement order.

[0049] Optionally, the corner point corresponding to the minimum value among multiple abscissas can be used as the first reference corner point.

[0050] Exemplarily, as Figure 3 shown in the right figure of 3, the abscissas of P1, P2, P3, and P4 are: x p12 , x p22 , x p32 , x p42 , since x p42 < x p12 < x p32 < x p22 , the first reference corner point can be P4.

[0051] Optionally, the smaller the slope corresponding to the remaining corner points, the higher the reference arrangement order. The calculation formula of the slope is as follows:

[0052]

[0053] Among them, y can represent the ordinate of the non-first reference corner point, x can represent the abscissa of the non-first reference corner point, y1 can represent the ordinate of the first reference corner point, and x1 can represent the abscissa of the first reference corner point.

[0054] Exemplarily, as shown in the right figure of 3, when the first reference corner point is P4, the slopes corresponding to P1, P2, and P3 can be K1, K2, and K3 respectively. Since K1 < K2 < K3, the reference sorting order can be: P1, P2, P3.

[0055] Optionally, the first reference corner point can be ranked first in the reference sorting, and then the remaining corner points can be ranked behind the first reference corner point according to the reference arrangement order to obtain the reference sorting.

[0056] Exemplarily, as shown in the right figure of 3, when the first reference corner point is P4 and the reference arrangement order is: P1, P2, P3, the reference sorting can be: P4, P1, P2, P3.

[0057] S130: Obtain the direction of the text area, and the direction represents the direction vector corresponding to the content in the text area.

[0058] Among them, as Figure 4 shown, obtaining the direction of the text area includes:

[0059] S131: Obtain the arrangement direction of the content in the text area, and the arrangement direction represents the reading order of the text.

[0060] As a way, the arrangement direction of the content in the text area can be obtained based on a direction recognition model, and the direction recognition model can be a convolutional neural network or the like.

[0061] S132: Obtain the direction vector of the text area based on the arrangement direction.

[0062] As a way, the direction vector of the text area can be obtained based on the vertical direction corresponding to the arrangement direction.

[0063] Exemplarily, as Figure 5 shown, Figure 5 the direction vector of the text area in the left figure of can be V 11 , Figure 5 the direction vector of the text area in the right figure of can be V 12 .

[0064] S133: Determine the direction of the text area based on the direction vector of the text area and a preset direction vector, where the preset direction vector represents the direction in the coordinate system corresponding to the image to which the text box belongs.

[0065] There may be multiple preset direction vectors, and the directions in the coordinate system represented by each preset direction vector may be vertically upward, vertically downward, horizontally left, and horizontally right.

[0066] As a method, we can first obtain the angles between the direction vector of the text area and the multiple preset direction vectors based on the direction vector of the text area and multiple preset direction vectors, and then compare the multiple angles with the preset ranges respectively. If there is an angle within the preset range, the direction represented by the preset direction vector corresponding to the angle is used as the direction of the text area.

[0067] For example, Figure 6 As shown, the direction vector of the text area can be V0, and the preset direction vector can be V 0° 、V 90° 、V 180° 、V 270° , the preset range can be -45° to 45°, and the direction of the text area can be vertically upward.

[0068] As another approach, an image containing a text region may be input into a recognition model to obtain the direction of the text region. The recognition model may be a convolutional neural network, etc.

[0069] S140: Obtain a target ordering of the plurality of corner points based on the directions and the reference ordering.

[0070] As a way, the reference ranking can be adjusted based on the orientation of the text region to obtain the target ranking of multiple corner points.

[0071] This embodiment provides a corner point sorting method. After obtaining a text region to be processed that includes multiple corner points, a reference sorting of the multiple corner points is obtained based on an angle representing the positional relationship between the text region and the edge of the image to which the text region belongs. The direction of a direction vector corresponding to the content in the text region is obtained, and a target sorting of the multiple corner points is obtained based on the direction and the reference sorting. Through the above method, a reference sorting of the multiple corner points can be obtained based on the angle representing the positional relationship between the text region and the edge of the image to which the text region belongs. A target sorting of the multiple corner points in the text region can then be obtained based on the direction of the direction vector corresponding to the content in the text region and the reference sorting. This allows downstream tasks to be executed directly based on the target sorting of the multiple corner points, thereby improving the efficiency of downstream tasks.

[0072] See also Figure 7 The present application provides a corner point sorting method, which is applied to electronic devices and includes:

[0073] S210: Acquire a text region to be processed, where the text region includes a plurality of corner points.

[0074] S220: Obtaining a reference ordering of the plurality of corner points based on an angle of the text region, wherein the angle represents a positional relationship between the text region and an edge of an image to which the text region belongs.

[0075] S230: Acquire a direction of the text region, where the direction represents a direction vector corresponding to the content in the text region.

[0076] S240: Obtain the corner point coordinates corresponding to each of the multiple corner points, where the corner point coordinates include a horizontal coordinate and a vertical coordinate.

[0077] As a method, the corner point coordinates corresponding to multiple corner points can be obtained based on a detection model (such as a convolutional neural network, etc.).

[0078] S250: Determine the midpoint of the text area based on the corner point coordinates corresponding to each of the multiple corner points.

[0079] The midpoint of the text area may refer to the intersection of the diagonals of the text area. For example, Figure 8 As shown, the midpoint of the text area may be c.

[0080] As a method, the abscissa and ordinate corresponding to each of the multiple corner points may be arithmetic averaged to obtain the abscissa and ordinate of the midpoint, thereby determining the midpoint of the text area.

[0081] S260: Based on the midpoint, the direction and the preset length, a midpoint vector is obtained, and the midpoint vector corresponds to an end point.

[0082] As one approach, the midpoint may be used as the starting point of the vector, and a point that is a preset distance away from the midpoint in the direction of the text area may be used as the end point of the vector to obtain the midpoint vector.

[0083] For example, Figure 8 As shown, the midpoint of the text area may be c, the direction may be vertically upward, the end point of the midpoint vector may be t, and the midpoint vector may be expressed as Vct.

[0084] S270: Obtain multiple sorting vectors based on the multiple corner points and the end point.

[0085] As one approach, multiple corner points may be used as starting points of corresponding sorting vectors, and the end point of the midpoint vector may be used as the end point of each sorting vector, so as to obtain multiple sorting vectors.

[0086] For example, Figure 8 As shown, the multiple corner points may be P1, P2, P3, and P4, respectively; the end point of the midpoint vector may be t; and the multiple sorting vectors may be V1, V2, V3, and V4, respectively.

[0087] S280: Obtain a first corner point based on the multiple sorting vectors and the midpoint vector, where the first corner point is the corner point ranked first in the target sorting.

[0088] As a way, such as Figure 9 As shown, obtaining a first corner point based on the multiple sorting vectors and the midpoint vector includes:

[0089] S281: Based on the multiple sorting vectors and the midpoint vector, obtain a first reference vector and a second reference vector, where the first reference vector and the second reference vector are sorting vectors located in a clockwise direction of the midpoint vector.

[0090] As a method, the midpoint vector can be cross-multiplied with multiple sorting vectors, and the first reference vector and the second reference vector can be obtained based on the result of the cross multiplication. If the result of the cross multiplication is greater than 0, it can be indicated that the corresponding sorting vector is located in the clockwise direction of the midpoint vector; if the result of the cross multiplication is less than 0, it can be indicated that the corresponding sorting vector is located in the counterclockwise direction of the midpoint vector. The midpoint vector can be expressed as (x1, y1), and the sorting vector can be expressed as (x2, y2). The formula for the cross multiplication is as follows:

[0091] r=x1·y2-x2·y1

[0092] Optionally, two sorted vectors may be selected from the sorted vectors whose cross product results are greater than 0 as the first reference vector and the second reference vector, respectively.

[0093] For example, Figure 10 As shown in the middle right figure, multiple sorting vectors can be translated toward the position of the midpoint vector so that the starting points of the multiple sorting vectors and the midpoint vector are the same. The translated multiple sorting vectors and the midpoint vector are then calculated according to the above cross product formula to obtain the first reference vector V4 and the second reference vector V1, respectively.

[0094] S282: Obtain a target vector based on the first reference vector and the second reference vector, where the target vector is the sorted vector farthest from the midpoint vector in the clockwise direction.

[0095] As one approach, the first reference vector and the second reference vector can be calculated using the cross product formula described above, and the target vector can be obtained based on the calculation result. During the calculation, the first reference vector can be expressed as (x1, y1), and the second reference vector can be expressed as (x2, y2). If the calculation result is greater than 0, it can be indicated that the second reference vector is located clockwise with respect to the first reference vector, and the second reference vector is used as the target vector; if the result of the cross product operation is less than 0, it can be indicated that the second reference vector is located counterclockwise with respect to the first reference vector, and the first reference vector is used as the target vector.

[0096] For example, Figure 10 As shown in the middle right figure, the first reference vector and the second reference vector can be V4 and V1 respectively. The first reference vector and the second reference vector are calculated according to the above cross product formula, and the target vector is the second reference vector V1.

[0097] S283: Taking the corner point corresponding to the target vector as the first corner point.

[0098] As a way, the corner point corresponding to the target vector can be used as the first corner point. For example, Figure 10 As shown in the middle right figure, the target vector can be the first reference vector, which corresponds to Figure 10 In the middle left picture, the first corner point can be P1.

[0099] S290: Obtain the target sorting based on the first corner points and the reference sorting.

[0100] As one approach, the first corner point can be placed first in the target order, and the remaining corner points can be arranged after the first corner point according to the reference order to obtain the target order. For example, the reference order can be P4, P1, P2, P3, the first corner point can be P1, and the target order can be P1, P2, P3, P4.

[0101] This embodiment provides a corner point sorting method, which, through the above-mentioned method, can obtain a reference sorting of multiple corner points based on the angle representing the positional relationship between the text region and the edge of the image to which the text region belongs, and then obtain a target sorting of multiple corner points in the text region based on the direction of the direction vector corresponding to the content in the text region and the reference sorting, so that downstream tasks can be directly executed based on the target sorting of multiple corner points, thereby improving the efficiency of downstream tasks. In addition, in this embodiment, the midpoint of the text region can be obtained based on the corner point coordinates corresponding to each of the multiple corner points, and then multiple sorting vectors and midpoint vectors can be obtained based on the multiple corner points and midpoints. The first corner point can be determined based on the multiple sorting vectors and midpoint vectors, so that the target sorting can be obtained based on the first corner point and the reference sorting, so that downstream tasks can be directly executed based on the target sorting of multiple corner points, thereby improving the efficiency of downstream tasks.

[0102] See also Figure 11 The present application provides a corner point sorting device 600, which runs on an electronic device. The device 600 includes:

[0103] The text region acquiring unit 610 is configured to acquire a text region, where the text region includes a plurality of corner points.

[0104] The reference order obtaining unit 620 is configured to obtain a reference order of the plurality of corner points based on an angle of the text region, wherein the angle represents a positional relationship between the text region and an edge of an image to which the text region belongs.

[0105] The direction acquiring unit 630 is configured to acquire the direction of the text region, where the direction represents a direction vector corresponding to the content in the text region.

[0106] The target ranking obtaining unit 640 is configured to obtain a target ranking of the plurality of corner points based on the directions and the reference ranking.

[0107] As a method, the reference sorting acquisition unit 620 is specifically used to obtain the corner point coordinates corresponding to each of the multiple corner points, and the corner point coordinates include a horizontal coordinate and a vertical coordinate; if the corner point coordinates corresponding to each of the multiple corner points have the same horizontal coordinate, it is determined that the angle represents that the text area is parallel to the edge of the image to which the text area belongs; if the horizontal coordinates corresponding to each of the multiple corner points are different, it is determined that the angle represents that the text area is not parallel to the edge of the image to which the text area belongs.

[0108] Optionally, the reference order obtaining unit 620 is specifically configured to, if the angle indicates that the text region is parallel to an edge of the image to which the text region belongs, add the horizontal coordinate and the vertical coordinate of each corner point to obtain a sum of coordinates corresponding to each of the multiple corner points; and determine a first reference corner point based on the multiple sums of coordinates, where the first reference corner point is the corner point ranked first in the reference order;

[0109] Based on the first reference corner point, a reference arrangement order of the remaining corner points in the plurality of corner points is determined; and based on the first reference corner point and the reference arrangement order, the reference ranking is obtained.

[0110] Optionally, the reference sorting acquisition unit 620 is specifically used to determine a first reference corner point based on the horizontal coordinates corresponding to each of the multiple corner points if the angle representing the text area is not parallel to the edge of the image to which the text area belongs, the first reference corner point being the corner point ranked first in the reference sorting; obtain the slopes of the straight lines corresponding to the remaining corner points in the multiple corner points, the straight lines being formed by connecting the remaining corner points in the multiple corner points with the first reference corner point respectively; determine the reference arrangement order of the remaining corner points in the multiple corner points based on the slopes; and obtain the reference sorting based on the first reference corner point and the reference arrangement order.

[0111] As a method, the direction acquisition unit 630 is specifically used to obtain the arrangement direction of the content in the text area, and the arrangement direction represents the reading order of the text; based on the arrangement direction, the direction vector of the text area is obtained; based on the direction vector of the text area and the preset direction vector, the direction of the text area is determined, and the preset direction vector represents the direction in the coordinate system corresponding to the image to which the text box belongs.

[0112] As a method, the target sorting acquisition unit 640 is specifically used to obtain the corner point coordinates corresponding to each of the multiple corner points, and the corner point coordinates include a horizontal coordinate and a vertical coordinate; based on the corner point coordinates corresponding to each of the multiple corner points, the midpoint of the text area is determined; based on the midpoint, the direction and the preset length, a midpoint vector is obtained, and the midpoint vector corresponds to an end point; based on the multiple corner points and the end point, multiple sorting vectors are obtained; based on the multiple sorting vectors and the midpoint vector, a first corner point is obtained, and the first corner point is the corner point ranked first in the target sorting; based on the first corner point and the reference sorting, the target sorting is obtained.

[0113] Optionally, the target sorting acquisition unit 640 is specifically used to obtain a first reference vector and a second reference vector based on the multiple sorting vectors and the midpoint vector, the first reference vector and the second reference vector being sorting vectors located in a clockwise direction of the midpoint vector; obtaining a target vector based on the first reference vector and the second reference vector, the target vector being the sorting vector farthest from the midpoint vector in a clockwise direction; and taking the corner point corresponding to the target vector as the first corner point.

[0114] The following will be combined Figure 12 An electronic device provided by this application is described.

[0115] See also Figure 12Based on the aforementioned corner point sorting method and apparatus, embodiments of the present application further provide another electronic device 100 capable of executing the aforementioned corner point sorting method. Electronic device 100 includes one or more (only one is shown in the figure) processors 102 and a memory 104 coupled to each other. The memory 104 stores a program capable of executing the aforementioned embodiments, and the processor 102 can execute the program stored in the memory 104.

[0116] The processor 102 may include one or more processing cores. The processor 102 utilizes various interfaces and circuits to connect various components within the electronic device 100. It executes instructions, programs, code sets, or instruction sets stored in the memory 104, and accesses data stored in the memory 104 to perform various functions and process data within the electronic device 100. Optionally, the processor 102 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 102 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 102 and may be implemented separately via a communication chip.

[0117] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the terminal 100 during use (such as a phone book, audio and video data, chat history data), etc.

[0118] Please refer to Figure 13, which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 800 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0119] The computer-readable storage medium 800 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in a suitable form.

[0120] In summary, the present application provides a corner point sorting method, device, and electronic device. After obtaining a text region to be processed that includes multiple corner points, the method obtains a reference sorting of the multiple corner points based on an angle representing the positional relationship between the text region and an edge of an image to which the text region belongs, obtains the direction of a direction vector corresponding to the content in the text region, and obtains a target sorting of the multiple corner points based on the direction and the reference sorting. Through the above-mentioned method, a reference sorting of the multiple corner points can be obtained based on the angle representing the positional relationship between the text region and an edge of an image to which the text region belongs, and then a target sorting of the multiple corner points in the text region can be obtained based on the direction of the direction vector corresponding to the content in the text region and the reference sorting. This allows downstream tasks to be performed directly based on the target sorting of the multiple corner points, thereby improving the efficiency of downstream tasks.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A corner point sorting method, characterized in that: The method comprises: Acquire a text region to be processed, where the text region includes a plurality of corner points; obtaining a reference ordering of the plurality of corner points based on an angle of the text region, wherein the angle represents a positional relationship between the text region and an edge of an image to which the text region belongs; Obtaining a direction of the text area, where the direction represents a direction vector corresponding to the content in the text area; Obtaining corner point coordinates corresponding to each of the multiple corner points, the corner point coordinates including a horizontal coordinate and a vertical coordinate; determining a midpoint of the text area based on the corner point coordinates corresponding to each of the multiple corner points; obtaining a midpoint vector based on the midpoint, the direction, and a preset length, the midpoint vector corresponding to an end point; obtaining multiple sorting vectors based on the multiple corner points and the end point; obtaining a first corner point based on the multiple sorting vectors and the midpoint vector, the first corner point being the corner point ranked first in the target sorting; The target ranking is obtained based on the first corner points and the reference ranking.

2. The method according to claim 1, characterized in that Before obtaining the reference order of the plurality of corner points based on the angle of the text area, the method includes: Obtaining the corner point coordinates corresponding to each of the multiple corner points, wherein the corner point coordinates include a horizontal coordinate and a vertical coordinate; If the corner point coordinates corresponding to the plurality of corner points have the same horizontal coordinate, determining that the angle represents that the text region is parallel to an edge of the image to which the text region belongs; If the horizontal coordinates corresponding to the multiple corner points are all different, determining the angle indicates that the text region is not parallel to the edge of the image to which the text region belongs.

3. The method according to claim 2, characterized in that The obtaining of a reference order of the plurality of corner points based on the angle of the text region includes: If the angle indicates that the text region is parallel to a side of the image to which the text region belongs, adding the horizontal coordinate and the vertical coordinate of each corner point to obtain the sum of the coordinates corresponding to each of the multiple corner points; Determine a first reference corner point based on the sum of the multiple coordinates, where the first reference corner point is the corner point ranked first in the reference order; Determining a reference arrangement order of the remaining corner points in the plurality of corner points based on the first reference corner point; The reference ranking is obtained based on the first reference corner point and the reference arrangement order.

4. The method according to claim 2, characterized in that The obtaining of a reference order of the plurality of corner points based on the angle of the text region includes: If the angle indicates that the text region is not parallel to an edge of the image to which the text region belongs, determining a first reference corner point based on the horizontal coordinates corresponding to each of the multiple corner points, the first reference corner point being the corner point ranked first in the reference order; Obtaining slopes of straight lines corresponding to the remaining corner points in the plurality of corner points, where the straight lines are formed by connecting the remaining corner points in the plurality of corner points and the first reference corner point; determining a reference arrangement order of remaining corner points in the plurality of corner points based on the slope; The reference ranking is obtained based on the first reference corner point and the reference arrangement order.

5. The method according to claim 1, wherein The obtaining of the direction of the text area includes: Obtaining an arrangement direction of content in the text area, where the arrangement direction represents a reading order of text; Based on the arrangement direction, obtaining a direction vector of the text area; The direction of the text region is determined based on the direction vector of the text region and a preset direction vector, where the preset direction vector represents the direction in a coordinate system corresponding to the image to which the text region belongs.

6. The method according to claim 1, characterized in that The obtaining of a first corner point based on the plurality of sorted vectors and the midpoint vector includes: Obtaining a first reference vector and a second reference vector based on the multiple sorting vectors and the midpoint vector, wherein the first reference vector and the second reference vector are sorting vectors located in a clockwise direction of the midpoint vector; Obtaining a target vector based on the first reference vector and the second reference vector, where the target vector is the sorted vector farthest from the midpoint vector in a clockwise direction; The corner point corresponding to the target vector is used as the first corner point.

7. A corner point sorting device, characterized in that: The device comprises: A text region acquiring unit, configured to acquire a text region, wherein the text region includes a plurality of corner points; a reference order obtaining unit, configured to obtain a reference order of the plurality of corner points based on an angle of the text region, wherein the angle represents a positional relationship between the text region and an edge of an image to which the text region belongs; a direction acquiring unit, configured to acquire a direction of the text region, wherein the direction represents a direction vector corresponding to the content in the text region; A target sorting acquisition unit is used to obtain the corner point coordinates corresponding to each of the multiple corner points, and the corner point coordinates include a horizontal coordinate and a vertical coordinate; based on the corner point coordinates corresponding to each of the multiple corner points, determine the midpoint of the text area; based on the midpoint, the direction and the preset length, obtain a midpoint vector, and the midpoint vector corresponds to an end point; based on the multiple corner points and the end point, obtain multiple sorting vectors; based on the multiple sorting vectors and the midpoint vector, obtain a first corner point, and the first corner point is the corner point ranked first in the target sorting; based on the first corner point and the reference sorting, obtain the target sorting.

8. An electronic device, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, wherein when the program code is run, the method according to any one of claims 1 to 6 is executed.

Images