Text box positioning method and apparatus, electronic device, and storage medium

Abstract

The application relates to a text box positioning method and device, electronic equipment and a storage medium. The positioning method comprises the following steps: detecting a first target in preset image data to obtain first image corresponding to the first target; processing the first image by using a feature extraction network to obtain first features; rasterizing the first features to generate first grid information; processing second grid information detected from the first grid information according to a preset feature merging rule to generate third grid information; and determining position information corresponding to the vertex of the text box and a positioning result according to the third grid information. Through the application, the problem that the processing time complexity of the text detection algorithm is high and the operation time is long when a text region is positioned in the related art is solved, the operation time of the text box region positioning is realized, the text box region is quickly and accurately positioned, and the beneficial effect of reducing power consumption is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a text box positioning method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the progress of computer technology and communication technology, automatic intelligent meter reading has gradually been used. In the related technology, by adding an embedded image acquisition and processing device to the meter, the original meter is not removed, and the meter image data is directly obtained remotely to complete remote meter reading, such as remote water meter reading.

[0003] In the related technology, to realize remote meter reading, the digital region of the related meter needs to be positioned to obtain the corresponding metering data, thereby completing meter reading. In the related technology, the digital region positioning is completed by text detection, and different text detection algorithms are used to realize text detection. However, in the related technology, when the text detection algorithm is used to position the text box region, the processing time complexity of the post-processing algorithm is high, and the operation time is long, which causes the intelligent meter to be unable to efficiently complete the remote meter reading task.

[0004] There is no effective solution to the problem that the processing time complexity of the text detection algorithm is high and the operation time is long when the text region is positioned in the related technology. SUMMARY

[0005] The present application provides a text box positioning method, device, electronic equipment and storage medium to at least solve the problem that the processing time complexity of the text detection algorithm is high and the operation time is long when the text region is positioned in the related technology.

[0006] In a first aspect, the present application provides a text box positioning method, comprising: detecting a first target in preset image data to obtain a first image corresponding to the first target; processing the first image by using a feature extraction network to obtain a first feature, wherein the feature extraction network is generated according to a second image and a text box corresponding to the second image, and the first feature is used to represent a text box corresponding to the first image; rasterizing the first feature to generate first grid information, and processing second grid information detected from the first grid information according to a preset feature merging rule to generate third grid information, wherein the first grid information is used to represent the position of a text element in the text box, and the second grid information includes the first grid information corresponding to the text element in the text box; and determining the position information of the vertex of the text box and the positioning result according to the third grid information.

[0007] In a second aspect, the present application provides a text box positioning device, comprising:

[0008] An acquisition module is configured to detect a first target in preset image data to obtain first image corresponding to the first target;

[0009] A feature extraction module is configured to process the first image by using a feature extraction network to obtain first feature, wherein the feature extraction network is generated according to a second image and a text box corresponding to the second image measured, and the first feature is used to represent the text box corresponding to the first image;

[0010] A processing module is configured to rasterize the first feature to generate first raster information, and process second raster information detected from the first raster information according to a preset feature merging rule to generate third raster information, wherein the first raster information is used to represent the position of a text element in the text box, and the second raster information includes the first raster information corresponding to the text element in the text box;

[0011] A calculation module is configured to determine position information corresponding to a vertex of the text box and a positioning result according to the third raster information.

[0012] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;

[0013] The memory is configured to store a computer program;

[0014] The processor is configured to execute the program stored on the memory to implement the steps of the text box positioning method of any one of the embodiments of the first aspect.

[0015] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps of the text box positioning method of any one of the embodiments of the first aspect.

[0016] Compared with the related art, the text box positioning method, device, electronic equipment and storage medium provided in the embodiment can detect a first target in preset image data to obtain first image corresponding to the first target, process the first image by using a feature extraction network to obtain first feature, rasterize the first feature to generate first raster information, process second raster information detected from the first raster information according to a preset feature merging rule to generate third raster information, and determine position information and a positioning result corresponding to a vertex of the text box according to the third raster information, thereby solving the problem of high processing time complexity and long operation time of a text detection algorithm in the related art when a text region is positioned, achieving the beneficial effects of shortening the operation time of text box region positioning in the process of remote meter reading of an intelligent meter, quickly and accurately positioning the text box region, and reducing power consumption.

[0017] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0020] Figure 1 A flowchart of a text box positioning method provided by an embodiment of the present application;

[0021] Figure 2 A flowchart of a text box positioning method provided by a preferred embodiment of the present application;

[0022] Figure 3 A structural block diagram of a text box positioning device according to an embodiment of the present application;

[0023] Figure 4 A structural diagram of an electronic equipment according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0025] Before the embodiments of the present application are described, the related technical means used by the text box positioning method of the embodiments of the present application and the problems existing in the related art are described as follows.

[0026] Commonly used algorithms in text detection include: target detection network Faster R-CNN (Faster Region CNN), scene text detection (Connectionist Text Proposal Network, CTPN), natural scene text detection (An Efficient and Accurate Scene Text Detector, EAST), and efficient text scene detection Advanced EAST; wherein, the text detection includes the following stages: candidate box extraction, candidate box filtering, bounding box regression, and candidate box merging, but EAST is only divided into the image segmentation classification network (Fully Convolutional Networks, FCN) generation text line parameter stage and the local perception non maximum suppression (non maximum suppression, NMS) stage, the network is simple, the detection accuracy and speed are high, in the present application, the efficient text scene detection Advanced EAST is adopted.

[0027] In the related art, Advance EAST uses a VGG (Visual Geometry Group) 16 convolutional neural network as a feature extraction network. If the resolution of the input image of the network is w*h, then the output dimension of the network is a feature vector of [h / 4, w / h, 7]. The output feature vector includes the following parameters: (1) a score map representing the probability of a text element being in a text box; (2) a vertex code including two channels, the first channel representing the confidence of whether the text element is a boundary element, and the second channel representing the probability of the point being a head / tail element; and (3) a vertex geo including four channels, respectively representing the predicted coordinates of the text element to the top-left and bottom-left vertices (or the top-right and bottom-right vertices). After the feature vector is extracted, in the related art, Advance EAST determines the text box element (trigger point) according to the output of the score map, then merges the upper and lower adjacent text box elements into a region list list, and then compares each text block list with the remaining text block list one by one. If there are left and right adjacent text box elements in the two text block lists, it is considered that the two text block lists belong to the same block group group. Finally, the four vertex coordinates of the text box are calculated. The specific processing flow includes the following: (1) determining the text box element (trigger point) according to the output of the score map; (2) traversing all the text box elements, and merging the left and right adjacent text box elements in the feature vector to form a plurality of text block lists list; (3) traversing all the text block lists list, and merging the upper and lower adjacent text block lists in the feature map into a block group group; (4) traversing the text box elements in the block group group, and determining the head / tail element according to the value output by the vertex code of the text box element; and (5) performing weighted average on the vertex geo predicted by the head (tail) element in each block group group to obtain the final text box vertex. In the related art, the time complexity of the post-processing algorithm is high during the algorithm processing process of the text box region positioning. Specifically, the upper and lower adjacent text box elements need to be merged into a text block list list, and then each text block list is compared with the remaining text block list one by one, so that the corresponding processing time complexity is O(n 2 ), where n represents the number of text block lists.

[0028] In the embodiments of the present application, based on the existing Advance EAST, the method of predicting after feature vector extraction is processed, the grid method is used to predict the vertex of the text box, and the processing workload generated by comparing each text block list with the remaining text block list one by one is omitted, so that the time complexity of the processing is reduced.

[0029] Figure 1 A flowchart of a text box positioning method provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the present application provides a text box positioning method, which includes the following steps: Figure 1

[0030] In step S101, a first target is detected in preset image data to obtain a first image corresponding to the first target.

[0031] In the present embodiment, the preset image data includes but is not limited to videos obtained from different application scenarios, for example, videos of remote meter reading controlled by digital intelligent metering, and can also be image data captured in different scenarios, for example, image photos taken.

[0032] In the present embodiment, when the preset image data corresponds to videos of remote meter reading, the image data is videos of a target metering table taken by a monitoring device (for example, a monitoring camera) in a target scenario. Meanwhile, the image data is acquired by an image acquisition module which acquires video frames one by one, so as to obtain images corresponding to the metering table. When acquiring the video frames, the image acquisition module acquires at least one video frame, so as to obtain a first image corresponding to a first target (a target metering table) to be detected for completing text box region positioning. In some optional embodiments, the target metering table corresponding to the first target includes but is not limited to a water meter, an electricity meter, and a gas meter.

[0033] In step S102, a first feature is obtained by processing the first image by using a feature extraction network, wherein the feature extraction network is generated according to a second image and a text box corresponding to the second image measured, and the first feature is used to represent a text box corresponding to the first image.

[0034] In the present embodiment, before the first image is processed by using the feature extraction network, a plurality of corresponding images (second images) of the same metering table are selected as training samples, and CNN training is performed. The trained CNN network can be used to extract the first feature corresponding to the first image.

[0035] ​In this embodiment, the feature extraction network includes but is not limited to a VGG16 network, and the first features extracted from the first image include the following parameters: a score map, a vertex code, and vertex coordinates vertex geo, wherein the score map is used to represent the confidence of the text element in the text box; the vertex code has two channels, the parameters of the first channel (the first list) of the vertex code represent the confidence of whether the text element is a boundary element, and the parameters of the second channel of the vertex code represent the probability of the text element being a head element or a tail element, wherein the default threshold 0-0.1 is a head element, and the default threshold 0.9-1 is a tail element; the vertex geo has four channels, respectively representing the four vertex coordinates predicted for the text element.

[0036] In this embodiment, the first features include the features of all text elements constituting the text box region, and all the text elements constitute the shape of the text box, but only the boundary text elements are used to regress and predict the vertex coordinates of the text box.

[0037] In step S103, the first features are rasterized to generate first raster information, and the second raster information detected from the first raster information is processed according to a preset feature merging rule to generate third raster information, wherein the first raster information is used to represent the position of the text element in the text box, and the second raster information includes the first raster information corresponding to the text element in the text box.

[0038] In this embodiment, after the first features corresponding to the first image are extracted, the related data corresponding to the first features is converted into the original recorded raster information in the raster table with the dimensions of the first features as the width and height; in this embodiment, the corresponding raster table is constructed by arranging the related data corresponding to the first features in the form of a raster table; at the same time, when the related data corresponding to the first features is arranged in the form of a raster table, if the dimensions of the first features are [w, h, c] and the dimensions of the text element confidence are [w, h, 1], a raster table with dimensions [w, h] is created, and the position of each text element in the first features is marked in the form of (x, y) coordinates in the raster table, for example: the position of the text element T1 in the first features is S1, the x coordinate of the text element T1 in the raster table is the remainder of S1 divided by w, and the y coordinate of the text element T1 in the raster table is the quotient of S1 divided by w. Of course, after the corresponding raster coordinates are marked, the original first features corresponding to the text element are stored in the corresponding raster, that is, the first raster information is generated.

[0039] In the embodiment, after the first raster information is generated, the corresponding revision parameters are generated according to the feature merging of the text elements, for example, the marks of the corresponding text elements at the start position and the end position of each list after the text elements are marked and merged into a list, and the group id marked on all the text elements in the group after the text elements are merged into a group; after the corresponding revision parameters are obtained, the corresponding revision parameters are written into the raster information corresponding to each text element in the second raster information, so as to generate the third raster information; when the text elements are merged according to the features, the positions of all the text elements arranged in the raster form are not operated, but a plurality of text elements are regionalized according to regions, and the regionalized information is recorded in the raster information corresponding to each text element, for example, it is assumed that the first raster information corresponding to a text element is (a, b, c, d), the text element is the start point of a list, which is marked as s, and the group corresponding to the text element is h, after the regionalization is completed, the raster information corresponding to the text element is (a, b, c, d, s, h); of course, when the raster information of the raster table is initially defined, the data structure corresponding to the finally generated raster information can be preset, and the values that do not initially appear can be set as empty; in the embodiment, each raster information of the third raster information corresponds to a data structure, and the feature information of the corresponding text element is determined through each state value and parameter value in the data structure.

[0040] In step S104, the position information and the positioning result corresponding to the vertex of the text box are determined according to the third raster information.

[0041] In the embodiment, each raster information of the third raster information can represent whether the text element belongs to the element in the text box, whether it is a boundary text element, whether it is a head or tail text element, and all the head boundary text elements used to predict the two vertices at the two ends of the short side of the head of the text box and all the tail boundary text elements used to predict the two vertices at the two ends of the short side of the tail of the text box are screened out through the corresponding raster information; then, the predicted vertex geo is weighted according to the corresponding boundary text element, so as to determine the position information corresponding to the vertex of the text box and determine the positioning result, and the positioning result is the position information corresponding to the vertex of the determined text box.

[0042] Through the steps S101 to S104, the first target is detected in the preset image data to obtain first image corresponding to the first target; the first image is processed by using the feature extraction network to obtain first feature; the first feature is rasterized to generate first grid information, and the second grid information detected from the first grid information is processed according to the preset feature merging rule to generate third grid information; and the position information and the positioning result corresponding to the vertex of the text box are determined according to the third grid information, thereby solving the problem of high processing time complexity and long operation time of the text detection algorithm in the related art when the text region is positioned, achieving the beneficial effects of shortening the operation time of the text box region positioning in the intelligent meter remote meter reading process, quickly and accurately positioning the text box region, and reducing power consumption.

[0043] It should be noted that, in the embodiment, by using the grid method, the corresponding processing is performed without using the existing method of comparing each list after generating the list to confirm the corresponding group, thereby reducing the dimension of the processing calculation amount and improving the processing efficiency and accuracy of the text box positioning.

[0044] It should be noted that, in the embodiment, the process step of determining the position information corresponding to the vertex of the text box after obtaining the third grid information corresponds to the process step of predicting the vertex coordinates of the text box completed by the Advance EAST in the prior art.

[0045] In some embodiments, the first feature includes score map information, element classification information, and vertex coordinate prediction information of the corresponding text element, and the rasterization of the first feature to generate the first grid information in step S103 can be implemented by the following steps:

[0046] Step 31, obtaining score map information from the first feature, wherein the score map information is used to represent the confidence of the text element being a text element in the text box.

[0047] Step 32, generating first marks corresponding to the text element according to the score map information, wherein the first marks are used to represent whether the text element is a text element in the text box.

[0048] In the embodiment, when the value corresponding to the score map information is greater than 0.9, it indicates that the text element belongs to the text element in the text box, and at this time, the value corresponding to the generated first mark is 1, otherwise, the value corresponding to the first mark is 0.

[0049] Step 33, generating first coordinates corresponding to the text element in the preset grid table according to the vertex coordinate prediction information, wherein the first grid information includes the first mark, the element classification information, the first coordinates, and the vertex coordinate prediction information.

[0050] In the embodiment, when the text element is a text element in the text box, the corresponding text element is arranged in the form of a grid table, the corresponding position relationship of the text element in the text box is marked in the form of grid coordinates; meanwhile, one text element corresponds to one grid, the first feature corresponding to each text element is converted into corresponding grid information, and the generated first mark and first coordinates are also added. In this way, the first feature is rasterized, and the rasterized grid information records the parameters corresponding to the original first feature, so as to complete the prediction of the vertex coordinates of the subsequent text box.

[0051] The score map information is obtained from the first feature in steps 31 to 32; the first mark corresponding to the text element is generated according to the score map information; and the first coordinates of the text element in the preset grid table are generated according to the vertex coordinate prediction information, wherein the first grid information includes the first mark, the element classification information, the first coordinates and the vertex coordinate prediction information, so as to rasterize the first feature, record the parameters corresponding to the original first feature in the rasterized grid information, and provide accurate data for the prediction of the vertex coordinates of the subsequent text box.

[0052] In some embodiments, the second grid information is detected from the first grid information, which can be achieved by the following steps: detecting the first mark in the first grid information, and determining the text element located in the text box according to the first mark, wherein the second grid information includes the first grid information corresponding to the text element located in the text box.

[0053] In the embodiment, the first grid information is further screened according to the first mark, so as to screen out the second grid information corresponding to the text element located in the text box, so as to ensure that the grid information corresponding to the text element subjected to the text element regionalization processing can complete the vertex prediction of the text box, thereby improving the processing efficiency and accuracy of the text box positioning.

[0054] In some embodiments, the second grid information detected from the first grid information is processed according to the preset feature merging rule in step S103 to generate third grid information, which can be achieved by the following steps:

[0055] In step 41, the first coordinates of the corresponding text element are obtained from the second grid information, and the text element blocks are merged based on the first coordinates to obtain a plurality of text block information, wherein the text block corresponding to the text block information includes a start text element and an end text element, the first coordinates of the start text element and the end text element have the same first coordinate value, and the text block information includes a preset block mark of the text elements in each text block.

[0056] In the embodiment, after obtaining the second grid information, the text elements need to be regionized to obtain the text regions. In the embodiment, after the text elements are arranged in the form of the grid table, the regionization process is performed. Instead of processing the arrangement positions of the text elements, the parameters after the regionization are written into the grid information of the corresponding text elements. For example, the text elements are marked as the block markers of a group, and whether the text elements are the starting point or the ending point of a list. Specifically, a list takes the text element with the coordinate (x1, y1) as the starting point and the text element with the coordinate (x2, y1) as the ending point. Then, the text element with the coordinate (x1, y1) is marked as the starting point of the list, and the text element with the coordinate (x2, y1) is marked as the ending point of the list. When the coordinate x1 or x2 is reached, the starting point or the ending point of the list is determined.

[0057] In the embodiment, the text elements corresponding to the starting point and the ending point of a list can be set according to a preset requirement. That is, the preset rule divides the text elements in the text box into multiple lists. After the rule is set, the corresponding starting point and ending point are determined.

[0058] In the embodiment, the text elements are merged into a list according to the left and right adjacent text elements. Therefore, in a list, the text elements correspond to the same row coordinate (the first coordinate value) in the grid table. Of course, in some optional embodiments, the text elements can also be merged according to the upper and lower adjacent text elements.

[0059] In the embodiment, after the text elements are merged into a list and the corresponding starting point and ending point are determined, a corresponding block marker is allocated to each list, that is, a group id is allocated.

[0060] In step 42, the text blocks corresponding to the multiple text block information are traversed, and whether the adjacent two text blocks intersect is determined according to the first coordinates of the starting point text element and the ending point text element of the text blocks.

[0061] In the embodiment, after the multiple lists are determined, the multiple lists need to be de-merged and merged. That is, two or more lists that need to be merged into one text block are merged, and a unified block marker is allocated after the merging. In the embodiment, whether the starting point text element and the ending point text element in the two lists adjacent in the upward direction intersect is determined, and whether the two lists need to be merged is determined according to the determination result.

[0062] In the case where it is judged that the two adjacent text blocks intersect, the block markers of the text elements in the two adjacent text blocks are updated to the same block marker, and the corresponding text block information is updated.

[0063] In the embodiment, in the case where it is judged that the two text blocks need to be merged, that is, the two lists need to be merged, the block markers corresponding to the two text blocks are updated to the same block marker. For example, if the two lists list0 and list1 intersect, and the block marker of list0 is 6 and the block marker of list1 is 5, the block markers of list0 and list1 are modified to be the same, that is, the block marker of list0 is modified to be 5.

[0064] In the embodiment, after the block marker is updated, the block marker is combined with the second grid information to generate the third grid information.

[0065] In the embodiment, after the block marker is updated, the block marker is combined with the second grid information to generate the third grid information.

[0066] In the embodiment, after the block marker is updated, the block marker is combined with the second grid information to generate the third grid information.

[0067] In some embodiments, the generating the third grid information based on the second grid information and the updated text block information in step 44 can be achieved by the following steps: obtaining the corresponding block markers from the updated text block information, and writing the obtained block markers into the second grid information corresponding to the text elements of each text block respectively to generate the third grid information, wherein the third grid information comprises the first marker, the element classification information, the first coordinates, the vertex coordinate prediction information and the block markers.

[0068] In the present embodiment, after determining that the corresponding text block needs to be merged, the block marker of the text block that needs to be modified is updated, the updated block marker is written into the second grid information corresponding to the corresponding text element, the generation of the third grid information is completed, and the text element whose block marker is modified can be determined through the third grid information. When traversing the text elements in the corresponding text block, the text elements are traversed according to the updated block marker, so as to complete the determination of the position of the vertex coordinate of the text box.

[0069] In some embodiments, the first coordinates comprise a first coordinate value and a second coordinate value, and the determining whether the two adjacent text blocks intersect according to the first coordinates corresponding to the start text element and the end text element of the text block in step 43 can be achieved by the following steps:

[0070] Step 431: detecting a first candidate coordinate and a second candidate coordinate from the first coordinates corresponding to all text elements of the two adjacent text blocks respectively, wherein the first candidate coordinate comprises the first coordinate of the start text element of the two adjacent text blocks, and the second candidate coordinate comprises the first coordinate of the end text element of the two adjacent text blocks.

[0071] In the present embodiment, when determining whether the text blocks intersect, the determination is made on the two text blocks adjacent in up and down directions, and the first coordinates of the start text elements and the end text elements of the two text blocks are correspondingly compared, that is, the first coordinates of the start text elements of the two text blocks are correspondingly compared, and the first coordinates of the end text elements of the two text blocks are correspondingly compared. Therefore, the first coordinates corresponding to the start text elements and the end text elements of the two text blocks are required as data objects.

[0072] Step 432: detecting a first target coordinate and a second target coordinate from the first candidate coordinate and the second candidate coordinate respectively, wherein the first target coordinate comprises the first coordinate with the maximum second coordinate value in the first candidate coordinate, and the second target coordinate comprises the first coordinate with the minimum second coordinate value in the second candidate coordinate.

[0073] In the embodiment, when determining whether the text blocks intersect, the determination is made on two adjacent text blocks in the vertical direction, and thus the second coordinate value (column coordinate) is used as the parameter for the coordinate determination.

[0074] In step 433, it is determined whether the second coordinate value of the first target coordinate is less than the second coordinate value of the second target coordinate, and if the second coordinate value of the first target coordinate is not less than the second coordinate value of the second target coordinate, it is determined that the two adjacent text blocks intersect.

[0075] In the embodiment, the two text blocks are denoted as list0 and list1. The maximum of the start point of list0 and the start point of list1 is taken as the first target coordinate, and the minimum of the end point of list0 and the end point of list1 is taken as the second target coordinate. If the second coordinate value of the second target coordinate is greater than or equal to the second coordinate value of the first target coordinate, it is determined that list0 and list1 intersect.

[0076] In the above steps 431 and 432, the first candidate coordinate and the second candidate coordinate are detected from the first coordinates corresponding to all text elements of the two adjacent text blocks, respectively. The first target coordinate and the second target coordinate are detected from the first candidate coordinate and the second candidate coordinate, respectively. It is determined whether the second coordinate value of the first target coordinate is less than the second coordinate value of the second target coordinate, and if the second coordinate value of the first target coordinate is not less than the second coordinate value of the second target coordinate, it is determined that the two adjacent text blocks intersect. Thus, the determination of whether the two adjacent text blocks intersect is realized. The determination of whether the two adjacent text blocks intersect is based on the coordinates of the text elements corresponding to the start point and the end point of the text blocks in the grid table, and thus it is not necessary to compare each text element in a list with the text elements in other lists, which reduces the dimension and the number of calculations in the text box positioning and improves the efficiency of the text box positioning.

[0077] In some embodiments, the determination of the position information corresponding to the vertex of the text box and the positioning result in step S104 according to the third grid information can be realized by the following steps.

[0078] In step 51, the third grid information corresponding to the text elements in the text block with the same block mark is traversed to obtain the corresponding element classification information, wherein the element classification information includes the head and tail element confidence and the boundary element confidence.

[0079] In the embodiment, the element classification information includes two channels, i.e., the head-tail element confidence and the boundary element confidence, wherein the first channel corresponds to the boundary element confidence, and the second channel corresponds to the head-tail element confidence; the head-tail element confidence is used to filter out the head text element used to predict the top-left and bottom-left vertexes of the text box and the tail text element used to predict the top-right and bottom-right vertexes of the text box; and the corresponding vertex coordinates are determined based on the filtered head text element or tail text element and the boundary element confidence.

[0080] In step 52, the first target text element and the second target text element are detected from all the text elements in the text block based on the head-tail element confidence, wherein the first target text element includes the text element with the head-tail element confidence greater than the first confidence threshold, and the second target text element includes the text element with the head-tail element confidence less than the second confidence threshold.

[0081] In the embodiment, when the head-tail element confidence of the corresponding text element is greater than 0.9, it is indicated that the text element belongs to the head text element; and when the head-tail element confidence of the corresponding text element is less than 0.1, it is indicated that the text element belongs to the tail text element.

[0082] In step 53, the vertex coordinate preset information corresponding to all the text elements in the first target text element and the corresponding boundary element confidence are weighted to obtain two first vertex coordinates corresponding to one side of the text box, and the vertex coordinate preset information corresponding to all the text elements in the second target text element and the corresponding boundary element confidence are weighted to generate two second vertex coordinates corresponding to the other side of the text box, wherein the position information corresponding to the vertexes of the text box includes the two first vertex coordinates and the two second vertex coordinates.

[0083] In the embodiment, the corresponding vertex coordinates are determined based on the first vertex coordinates corresponding to the filtered head text element or tail text element and the boundary element confidence.

[0084] In the embodiment, the vertex geo predicted by the head text element or tail text element in each text block is weighted and averaged to obtain the four vertex coordinates of the text box of the meter.

[0085] In the embodiment, the prediction of the four vertex coordinates of the text box is performed according to the following formula:

[0086]

[0087] wherein quad(pv) represents the corresponding vertex coordinates, n represents the number of boundary text elements in the current text box, score i represents the boundary element confidence of the current text element, and pvi are four vertex coordinates of the current text element predicted by the feature extraction network.

[0088] Figure 2 A flowchart of a text box positioning method provided for the preferred embodiment of the present application is shown in FIG. 2. As shown in FIG. 2, the method flow includes the following steps: Figure 2

[0089] Step S201, collect the meter picture, and then perform step S202.

[0090] Step S202, extract the first feature using the trained feature extraction network, and then perform step S203.

[0091] Step S203, construct a grid table according to the width and height of the feature vector corresponding to the first feature, and then perform step S204.

[0092] Step S204, determine the text elements in the text box according to the values output by the score map corresponding to the first feature, and mark the positions of the text elements in each text box in the grid table, and then perform step S205.

[0093] Step S205, traverse all the text elements in the text box, and merge the text elements in the left and right adjacent text boxes into a text block, mark the start point and end point of each text block in the grid table, and assign a block label to each text block, and then perform step S206.

[0094] Step S206, traverse all the text blocks, compare the coordinates of the upper and lower adjacent text blocks, if they intersect, modify the block label of the corresponding text block, and then perform step S207.

[0095] Step S207, traverse the text elements in the text box in the text block, and determine the corresponding head text element or tail text element according to the head and tail element confidence, and then perform step S208.

[0096] Step S208, perform weighted average on the vertex geo predicted by the head text element or tail text element in each text block to obtain the four vertex coordinates of the text box of the meter.

[0097] In the embodiment, a text box positioning device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and details have been described above. The terms "module", "unit", "sub-unit" and the like used below can be a combination of software and / or hardware that can implement a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware can also be implemented and conceived. ​

[0098] Figure 3 is a structural block diagram of a positioning device of a text box according to an embodiment of the present application, as shown in the figure, the device comprises: Figure 3

[0099] The acquisition module 31 is configured to detect a first target in preset image data to obtain a first image corresponding to the first target.

[0100] The feature extraction module 32 is coupled to the acquisition module 31 and configured to process the first image by using a feature extraction network to obtain first features, wherein the feature extraction network is generated by training according to a second image and a text box corresponding to the second image, and the first features are used to represent the text box.

[0101] The processing module 33 is coupled to the feature extraction module 32 and configured to rasterize the first features to generate first raster information, and process second raster information detected from the first raster information according to a preset feature merging rule to generate third raster information, wherein the first raster information is used to represent the position of a text element in the text box, and the second raster information includes the first raster information corresponding to the text element in the text box.

[0102] The calculation module 34 is coupled to the processing module 33 and configured to determine position information corresponding to the vertex of the text box and a positioning result according to the third raster information.

[0103] By using the device according to the embodiments of the present application, the first target is detected in the preset image data to obtain the first image corresponding to the first target, the first image is processed by using the feature extraction network to obtain the first features, the first features are rasterized to generate the first raster information, the second raster information detected from the first raster information is processed according to the preset feature merging rule to generate the third raster information, and the position information corresponding to the vertex of the text box and the positioning result are determined according to the third raster information, thereby solving the problem of high processing time complexity and long operation time of the text detection algorithm in the related art when the text area is positioned, achieving the beneficial effects of shortening the operation time of the text box area positioning in the remote meter reading process of the intelligent meter, quickly and accurately positioning the text box area, and reducing the power consumption.

[0104] In some embodiments, the processing module 33 can further include:

[0105] The first acquisition unit is configured to acquire score map information from the first features, wherein the score map information is used to represent the confidence of the text element being the text element in the text box.

[0106] ​The first determining unit is coupled to the first obtaining unit and configured to generate a first mark corresponding to the text element according to the score map information, where the first mark is used to represent whether the text element is a text element in the text box.

[0107] The first generating unit is coupled to the first determining unit and configured to generate first coordinates of the text element in a preset grid table according to the vertex coordinate prediction information, where the first grid information includes the first mark, the element classification information, the first coordinates and the vertex coordinate prediction information.

[0108] In some embodiments, the text box positioning apparatus is further configured to detect the first mark in the first grid information and determine the text element located in the text box according to the first mark, where the second grid information includes the first grid information corresponding to the text element located in the text box.

[0109] In some embodiments, the processing module 33 further includes:

[0110] The second obtaining unit is configured to obtain the first coordinates of the corresponding text element from the second grid information and perform text element block merging based on the first coordinates to obtain a plurality of text block information, where the text block corresponding to the text block information includes a start text element and an end text element, the first coordinates of the start text element and the end text element have the same first coordinate value, and the text block information includes a preset block mark of the text element in each text block.

[0111] The second determining unit is coupled to the second obtaining unit and configured to traverse the text blocks corresponding to the plurality of text block information and determine whether two adjacent text blocks intersect according to the first coordinates of the start text element and the end text element of the text blocks.

[0112] The second processing unit is coupled to the second determining unit and configured to, in the case where it is determined that the two adjacent text blocks intersect, update the block marks of the text elements in the two adjacent text blocks to the same block mark and update the corresponding text block information.

[0113] The second generating unit is coupled to the second processing unit and configured to generate third grid information based on the second grid information and the updated text block information.

[0114] In some embodiments, the second generating unit is further configured to obtain the corresponding block mark from the updated text block information and write the obtained block mark into the second grid information corresponding to the text element of each text block respectively to generate the third grid information, where the third grid information includes the first mark, the element classification information, the first coordinates, the vertex coordinate prediction information and the block mark.

[0115] In some embodiments, the first coordinate comprises a first coordinate value and a second coordinate value, and the second judging unit is further configured to: detect, in step 431, a first candidate coordinate and a second candidate coordinate from the first coordinates corresponding to all text elements of the two adjacent text blocks, respectively, wherein the first candidate coordinate comprises the first coordinate of the start text element of the two adjacent text blocks, and the second candidate coordinate comprises the first coordinate of the end text element of the two adjacent text blocks; detect a first target coordinate and a second target coordinate from the first candidate coordinate and the second candidate coordinate, respectively, wherein the first target coordinate comprises the first coordinate with the largest second coordinate value in the first candidate coordinate, and the second target coordinate comprises the first coordinate with the smallest second coordinate value in the second candidate coordinate; and determine that the two adjacent text blocks intersect when the second coordinate value of the first target coordinate is smaller than the second coordinate value of the second target coordinate.

[0116] In some embodiments, the calculation module further comprises:

[0117] a third processing unit configured to traverse the third grid information corresponding to the text elements in the text block with the same block label to obtain corresponding element classification information, wherein the element classification information comprises a head-tail element confidence and a boundary element confidence;

[0118] a first detecting unit, coupled to the third processing unit, configured to detect a first target text element and a second target text element from all text elements in the text block based on the head-tail element confidence, wherein the first target text element comprises a text element with a head-tail element confidence greater than a first confidence threshold, and the second target text element comprises a text element with a head-tail element confidence less than a second confidence threshold;

[0119] a first calculating unit, coupled to the first detecting unit, configured to perform weighted calculation on the vertex coordinate preset information corresponding to all text elements in the first target text element and the corresponding boundary element confidence to obtain two first vertex coordinates corresponding to one side of the text box, and perform weighted calculation on the vertex coordinate preset information corresponding to all text elements in the second target text element and the corresponding boundary element confidence to generate two second vertex coordinates corresponding to the other side of the text box, wherein the position information corresponding to the vertex of the text box comprises the two first vertex coordinates and the two second vertex coordinates.

[0120] Figure 4 is a structural schematic diagram of an electronic device of an embodiment of the present application, such as Figure 4As shown, the embodiment of the present application provides an electronic device, comprising a processor 41, a communication interface 42, a memory 43 and a communication bus 44, wherein the processor 41, the communication interface 42 and the memory 43 complete mutual communication through the communication bus 44,

[0121] The memory 43 is used for storing a computer program.

[0122] The processor 41 is used for executing the program stored in the memory 43, so as to realize the method steps in the method. Figure 1 The processing in the server realizes the method steps in the method, and the technical effects brought by the positioning method of the Chinese text box are the same as those of the above-mentioned embodiment, which will not be described here.

[0123] Figure 1 The processing in the server realizes the method steps in the method, and the technical effects brought by the positioning method of the Chinese text box are the same as those of the above-mentioned embodiment, which will not be described here. Figure 1 The communication bus mentioned in the above terminal can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation,

[0124] Figure 3 In the above-mentioned terminal, only one thick line is used to represent the communication bus, but it does not mean that there is only one bus or only one type of bus.

[0125] The communication interface is used for communication between the above-mentioned terminal and other devices.

[0126] The memory can include a random access memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0127] ​​The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0128] The embodiment of the present application further provides a computer readable storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the text box positioning method provided by any one of the preceding method embodiments.

[0129] In another embodiment provided by the present application, a computer program product containing instructions, which, when executed on a computer, causes the computer to perform the steps of the text box positioning method described in any one of the preceding embodiments.

[0130] It should be noted that, in the present document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between or among the entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. In other words, "comprising" does not exclude the presence of additional elements.

[0131] The above description is merely one specific implementation of the application. Many modifications and other embodiments of the application set forth herein will come to mind to one skilled in the art to which the application pertains having the benefit of the teachings presented in the foregoing description. Therefore, it is to be understood that the application is not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the spirit and scope of the application. Moreover, although the foregoing description has been described in the context of particular embodiments, it is not intended that such embodiments represent a limit to the scope of the application.

Claims

1. A method of positioning a text box, characterized by, Comprise: Detecting a first target in preset image data to obtain a first image corresponding to the first target; Processing the first image by using a feature extraction network to obtain a first feature, wherein the feature extraction network is generated according to a second image and a text box corresponding to the second image measured, and the first feature is used to represent a text box corresponding to the first image; Griding the first feature to generate first grid information, processing second grid information detected from the first grid information according to a preset feature merging rule to generate third grid information, wherein the first grid information is used to represent the position of a text element in the text box, and the second grid information includes the first grid information corresponding to the text element in the text box; According to the third grid information, determining the position information corresponding to the vertex of the text box and the positioning result; Wherein, the first feature includes score map information, element classification information and vertex coordinate prediction information of the corresponding text element, the first feature is gridded to generate the first grid information, including: obtaining the score map information from the first feature, wherein the score map information is used to represent the confidence of the text element as a text element in the text box; according to the score map information, generating a first mark corresponding to the text element, wherein the first mark is used to represent whether the text element is a text element in the text box; according to the vertex coordinate prediction information, generating a first coordinate corresponding to the text element in the preset grid table, wherein the first grid information includes the first mark, the element classification information, the first coordinate and the vertex coordinate prediction information; Wherein, the second grid information is detected from the first grid information, including: detecting the first mark in the first grid information, and determining the text element located in the text box according to the first mark, wherein the second grid information includes the first grid information corresponding to the text element located in the text box; The second grid information detected from the first grid information is processed according to a preset feature merging rule to generate third grid information, including: obtaining the first coordinates corresponding to the text elements from the second grid information, and performing text element block merging based on the first coordinates to obtain a plurality of text block information, wherein the text block corresponding to the text block information includes a starting text element and an ending text element, the first coordinates of the starting text element and the ending text element have the same first coordinate value, and the text block information includes a preset block marker of the text elements in each text block; traversing the text blocks corresponding to the plurality of text block information, and judging whether two adjacent text blocks intersect according to the first coordinates of the starting text element and the ending text element of the text blocks; in the case of judging that the two adjacent text blocks intersect, updating the block markers of the text elements in the two adjacent text blocks to the same block marker, and updating the corresponding text block information; generating the third grid information based on the second grid information and the updated text block information; The third grid information is generated based on the second grid information and the updated text block information, including: obtaining the block markers corresponding to the updated text block information, and writing the obtained block markers into the second grid information corresponding to the text elements of each text block respectively to generate the third grid information, wherein the third grid information includes the first marker, the element classification information, the first coordinate, the vertex coordinate prediction information and the block marker.

2. The method of claim 1, wherein, The first coordinate includes the first coordinate value and the second coordinate value, and whether two adjacent text blocks intersect is judged according to the first coordinates corresponding to the starting text element and the ending text element of the text blocks, including: Detecting a first candidate coordinate and a second candidate coordinate from the first coordinates corresponding to all text elements of two adjacent text blocks respectively, wherein the first candidate coordinate includes the first coordinates of the starting text elements of the two adjacent text blocks, and the second candidate coordinate includes the first coordinates of the ending text elements of the two adjacent text blocks; Detecting a first target coordinate and a second target coordinate from the first candidate coordinate and the second candidate coordinate respectively, wherein the first target coordinate includes the first coordinate with the maximum second coordinate value in the first candidate coordinate, and the second target coordinate includes the first coordinate with the minimum second coordinate value in the second candidate coordinate; Judging whether the second coordinate value of the first target coordinate is less than the second coordinate value of the second target coordinate, and determining that the two adjacent text blocks intersect in the case of judging that the second coordinate value of the first target coordinate is not less than the second coordinate value of the second target coordinate.

3. The method of claim 1, wherein, The position information corresponding to the vertex of the text box and a positioning result are determined according to the third grid information, and the method comprises the following steps: The third grid information corresponding to the text elements in the text block marked with the same block mark is traversed to obtain corresponding element classification information, wherein the element classification information comprises head and tail element confidence and boundary element confidence; Based on the head and tail element confidence, first target text elements and second target text elements are detected from all text elements in the text block, wherein the first target text elements comprise the text elements with head and tail element confidence greater than a first confidence threshold, and the second target text elements comprise the text elements with head and tail element confidence less than a second confidence threshold; The vertex coordinate preset information corresponding to all text elements in the first target text elements and the corresponding boundary element confidence are weighted to obtain two first vertex coordinates corresponding to one side of the text box, and the vertex coordinate preset information corresponding to all text elements in the second target text elements and the corresponding boundary element confidence are weighted to generate two second vertex coordinates corresponding to the other side of the text box, wherein the position information corresponding to the vertex of the text box comprises the two first vertex coordinates and the two second vertex coordinates.

4. A text box positioning apparatus, characterized by comprising: The method comprises the following steps: The first target is detected in the preset image data to obtain first image corresponding to the first target; The first image is processed by using a feature extraction network to obtain first feature, wherein the feature extraction network is generated by training a second image and a text box corresponding to the second image measured, and the first feature is used to represent the text box corresponding to the first image; The first feature is rasterized to generate first grid information, and the second grid information detected from the first grid information is processed according to a preset feature merging rule to generate third grid information, wherein the first grid information is used to represent the position of the text element in the text box, and the second grid information comprises the first grid information corresponding to the text element in the text box; The position information corresponding to the vertex of the text box and a positioning result are determined according to the third grid information; The processing module comprises: a first acquisition unit, configured to acquire score map information from the first feature, wherein the score map information is used to represent the confidence of the text element being a text element in the text box; a first judgment unit, coupled with the first acquisition unit, configured to generate first marks corresponding to the text element according to the score map information, wherein the first mark is used to represent whether the text element is a text element in the text box; and a first generation unit, coupled with the first judgment unit, configured to generate first coordinates corresponding to the text element in a preset grid table according to vertex coordinate prediction information, wherein the first grid information comprises the first mark, element classification information, first coordinates and vertex coordinate prediction information. The text box positioning apparatus is further configured to detect a first mark in the first grid information and determine a text element located in the text box according to the first mark, wherein the second grid information comprises the first grid information corresponding to the text element located in the text box. The processing module comprises: a second acquisition unit, configured to acquire the first coordinates of the corresponding text element from the second grid information, and perform text element block merging based on the first coordinates to obtain a plurality of text block information, wherein the text block information corresponding to the text block comprises a start text element and an end text element, the first coordinates of the start text element and the end text element have the same first coordinate value, and the text block information comprises text elements in each text block having a preset block mark; a second judgment unit, coupled with the second acquisition unit, configured to traverse the text blocks corresponding to the plurality of text block information, and judge whether two adjacent text blocks intersect according to the first coordinates of the start text element and the end text element of the text blocks; a second processing unit, coupled with the second judgment unit, configured to, in the case that the two adjacent text blocks are judged to intersect, update the block marks of the text elements in the two adjacent text blocks to the same block mark, and update the corresponding text block information; and a second generation unit, coupled with the second processing unit, configured to generate third grid information based on the second grid information and the updated text block information. The second generation unit is configured to acquire the corresponding block marks from the updated text block information, and write the acquired block marks into the second grid information corresponding to the text elements of each text block respectively to generate the third grid information, wherein the third grid information comprises the first mark, the element classification information, the first coordinates, the vertex coordinate prediction information, and the block marks.

5. An electronic device, comprising: The computer program is executed by the processor to implement the text box positioning method of any one of claims 1-3. The computer program is executed by the processor to implement the text box positioning method of any one of claims 1-3. The computer program is executed by the processor to implement the text box positioning method of any one of claims 1-3.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, ​

Images