Data processing method and device

Abstract

This specification discloses a data processing method and device. After acquiring an image, the method first determines the character strings corresponding to the text lines in the image whose area falls within the same designated area of ​​the image as associated character strings. Then, based on pre-stored correspondences between data identifiers and geographic locations, the method determines the data identifier corresponding to the geographic location at the time the image was acquired. Then, from the determined data identifiers, a data identifier that matches at least one character string is determined as an update identifier. Finally, based on other character strings associated with the character strings matching the update identifier, the data corresponding to the update identifier is updated. By determining the associated relationships between the character strings contained in the acquired image and performing data updates, a new data update method is implemented, reducing the difficulty of data supplementation.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a data processing method and device. Background Art

[0002] With the development of delivery services, delivery platforms are increasingly covering a wide range of areas. To better execute their business, delivery platforms need to supplement supplier information. For example, to execute delivery services, delivery platforms require at least the supplier's location, name, and available delivery items. However, other information, such as order phone numbers, addresses, zip codes, and business hours, may not be readily available and require supplementary information.

[0003] However, the distribution platform has difficulties in collecting supplier information, and there is a problem of incomplete collection of supplier information. Summary of the Invention

[0004] The embodiments of this specification provide a data processing method and apparatus to partially solve the problems existing in the prior art.

[0005] The embodiments of this specification adopt the following technical solutions:

[0006] The data processing methods provided in this manual include:

[0007] Acquire images;

[0008] determining an area in the image where a text line is located and at least one designated area in the image;

[0009] Determine the character strings corresponding to the text lines whose regions fall within the same designated region as the character strings having an associated relationship;

[0010] Determining the data identifier corresponding to the geographical location when the image was collected, based on the geographical location when the image was collected and the pre-stored correspondence between the data identifiers and the geographical locations;

[0011] Among the determined data identifiers, determining a data identifier that matches at least one character string as an update identifier;

[0012] The data corresponding to the update identifier is updated according to other character strings that are associated with the character string that matches the update identifier.

[0013] Optionally, acquiring an image specifically includes:

[0014] The images collected by the delivery transport at the supplier's location when the delivery transport is performing a delivery task are acquired, wherein the images at least include the storefront of the supplier.

[0015] Optionally, determining the geographical location when the image is collected specifically includes:

[0016] Determine the geographical location when the image was collected based on the coordinates of the POI provided by the supplier; or

[0017] Determine the location information of the acquisition device when the distribution transport capacity acquires the image through the acquisition device.

[0018] Optionally, character strings corresponding to text lines whose regions fall within the same designated region are determined as character strings having an associated relationship, specifically including:

[0019] Determining the character strings corresponding to the respective texts according to the regions where the text lines are located in the image;

[0020] Determining the regions where the text lines are located that fall within the same designated region according to the positions of the designated regions in the image and the positions of the regions where the text lines are located in the image;

[0021] Determine the association relationship between the character strings corresponding to the regions where each text line falls within the same specified region.

[0022] Optionally, determining a data identifier corresponding to a geographical location when the image is collected specifically includes:

[0023] Determining, from pre-stored geographical locations, geographical locations that are less than a preset distance from the geographical location when the image was captured;

[0024] According to the pre-stored correspondence between each data identifier and each geographical location, the data identifier corresponding to each geographical location that is less than the preset distance is determined to be the data identifier corresponding to the geographical location when the image was collected.

[0025] Optionally, updating the data corresponding to the update identifier according to other character strings associated with the character string matching the update identifier specifically includes:

[0026] Determining the data corresponding to the update identifier based on the pre-stored correspondence between each data and each geographical location and the geographical location corresponding to the update identifier;

[0027] The data corresponding to the update identifier is updated according to other character strings that are associated with the character string that matches the update identifier.

[0028] Optionally, updating the data corresponding to the update identifier specifically includes:

[0029] Determining, from other character strings associated with the character string matching the update identifier, a character string that is inconsistent with the data corresponding to the update identifier;

[0030] The determined character string is added to the data corresponding to the update identifier.

[0031] The data processing device provided in this specification includes:

[0032] Acquisition module, acquires images;

[0033] A first determining module determines an area in the image where a text line is located and at least one designated area in the image;

[0034] A relationship determination module determines the character strings corresponding to the text lines whose regions fall into the same designated region as the character strings having an associated relationship;

[0035] an identifier determination module, which determines a data identifier corresponding to the geographical location when the image was collected, based on the geographical location when the image was collected and a pre-stored correspondence between each data identifier and each geographical location;

[0036] A matching module, which determines, among the determined data identifiers, a data identifier that matches at least one character string as an update identifier;

[0037] The data processing module updates the data corresponding to the update identifier according to other character strings that are associated with the character string that matches the update identifier.

[0038] The electronic device provided in this specification includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned data processing method is implemented.

[0039] The computer-readable storage medium provided in this specification stores a computer program, and when the computer program is executed by a processor, the above-mentioned data processing method is implemented.

[0040] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0041] After acquiring an image, the character strings corresponding to the text lines in the image whose regions fall within the same designated region of the image are first determined as associated character strings. Then, based on pre-stored correspondences between data identifiers and geographic locations, the data identifier corresponding to the geographic location at the time the image was captured is determined. From the determined data identifiers, a data identifier that matches at least one character string is identified as an update identifier. Finally, based on other character strings associated with the character strings matching the update identifier, the data corresponding to the update identifier is updated. By determining the associated relationships among the character strings contained in the captured image and determining the character strings for data update, a novel data update method is implemented, reducing the difficulty of data supplementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 A schematic diagram of the data processing process provided in the embodiments of this specification;

[0044] Figure 2 A schematic diagram of a designated area provided in an embodiment of this specification;

[0045] Figure 3 A schematic diagram of the area where text lines falling into the same designated area are provided in an embodiment of this specification;

[0046] Figure 4 A schematic diagram of determining data identification provided in an embodiment of this specification;

[0047] Figure 5 A schematic diagram of the minimum bounding rectangle determined by the prior art;

[0048] Figure 6 A schematic diagram of determining the vertical direction of text provided in an embodiment of this specification;

[0049] Figure 7 A wiring diagram provided for the embodiments of this specification;

[0050] Figure 8 A schematic diagram of the circumscribed quadrilateral provided for this specification;

[0051] Figure 9 and Figure 10 A schematic diagram of a process for determining the area where a text line is located provided in an embodiment of this specification;

[0052] Figure 11A schematic diagram of the structure of a data processing device provided in an embodiment of this specification;

[0053] Figure 12 This is a schematic structural diagram of an electronic device for implementing a data processing method provided in an embodiment of this specification. DETAILED DESCRIPTION

[0054] To make the purpose, technical solutions, and advantages of this specification more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0055] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0056] Figure 1 The data processing process provided in the embodiments of this specification may specifically include the following steps:

[0057] S100: Acquire an image.

[0058] In this specification, the data processing process can be executed by the terminal, or the server can receive the image uploaded by the terminal and execute the data processing process, wherein the terminal may include: mobile phones, tablet computers and other devices, the server can be a single server, or a system composed of multiple servers, such as a distributed server. This specification does not limit the specific device that executes the image recognition process, and it can be set as needed.

[0059] For ease of description, this specification uses the example of a delivery platform supplementing store information based on images captured by a terminal, assuming that the platform needs to supplement supplier information. For the delivery platform, suppliers are stores that provide delivery items. For example, a food delivery platform's suppliers could be restaurants offering takeout food. Therefore, in the embodiments presented in this specification, the delivery platform's server can obtain images containing storefronts.

[0060] In one or more embodiments provided in this specification, the delivery platform may require the delivery transport to capture images of the supplier's storefront when performing a delivery task. Therefore, when performing a delivery task, the delivery transport may capture images containing the supplier's storefront through the delivery transport terminal and upload them to the delivery platform's server, which obtains images containing the storefront.

[0061] Of course, this specification does not limit when the image is obtained. For example, the server can obtain the image from images of storefronts collected historically, or obtain images uploaded by the distribution terminal in real time.

[0062] Furthermore, in this specification, the image containing the storefront can also be captured and uploaded by another user's terminal. For example, if a user uploads a store environment image that contains the storefront, or if a store operator uploads an image of their own storefront to the server, the server can also obtain the image used for supplementary data from these user-uploaded images.

[0063] S102: Determine an area in the image where a text line is located and at least one designated area in the image.

[0064] In this specification, after acquiring an image, the server can determine, based on the image, an area corresponding to at least one text line in the image and at least one designated area, wherein the area corresponding to the text line is the area where the text line is located in the image determined by the image segmentation model, and the designated area is the area where the store sign in the image is located. Of course, depending on the application scenario, the designated area can also be the area where a store billboard, bulletin board, etc. is located, or the designated area can be the area where the store's facade is located, in which case the server can determine the designated area by training a corresponding model. This specification does not specifically limit the designated area. In the scenario where the server acquires an image containing a store's facade in order to update store information, it is sufficient as long as different designated areas correspond to different stores.

[0065] Figure 2 The designated area diagram provided for this manual may use the area where the store's billboard is located as the designated area, or the area where the store's sign is located as the designated area, or the area where the store's storefront is located as the designated area.

[0066] Specifically, since it is necessary to distinguish between the areas belonging to text lines and the areas belonging to non-text lines in the image, the server can first input the image into a pre-trained image segmentation model to obtain the image segmentation result output by the image segmentation model. The image segmentation result can be as follows: Figure 3The polygons corresponding to the text lines and the areas of non-text lines obtained by segmenting the image are shown, and then the minimum connected area of ​​the polygons corresponding to each text line is determined to determine the minimum bounding rectangle of each text line in the image as the area where the text line is located in the image. Among them, the image segmentation model can specifically be a full convolutional network model (Fully Convolutional Networks, FCN), a conditional random field (Conditional Random Field, CRF), etc., which is not limited in this specification. For the convenience of description, the FCN model will be used as an example for subsequent explanation. The area where the text lines are located output by the FCN model is a regular polygon, and the polygon can be further processed to determine the minimum bounding rectangle of each text line. It should be noted that since segmenting the image by the FCN model and determining the minimum bounding rectangle of the text line is already a relatively mature technology, this specification will not elaborate on the use of the FCN model, the training process of the FCN, and the process of determining the minimum bounding rectangle.

[0067] Similarly, the designated area determined from the image can also be determined using a trained FCN model or other image detection models. Of course, the FCN model for determining the designated area and the FCN for segmenting the text line can be different models trained through different training samples, that is, by training two models separately, they are used to determine the area where the text line in the image is located and the designated area. In addition, a model that can simultaneously segment the text line and the designated area can be trained. The server can input the image into a model to determine the area where the text line in the image is located and the designated area. For example, the current FCN image can simultaneously identify the areas where the airplane, ground, and sky are located in an image. Similarly, a model that can simultaneously identify the area where the text line is located and the designated area in the image can be trained through suitable samples.

[0068] S104: Determine the character strings corresponding to the text lines whose regions fall within the same designated region as the character strings having an associated relationship.

[0069] In this specification, when a server acquires an image containing a storefront to update store information, different designated areas correspond to different stores. Therefore, the strings corresponding to the text lines within the same designated area in the image should also correspond to the same store. Therefore, after the server determines the areas within each text line in the image and the designated areas, it can identify the strings corresponding to the text lines within the same designated area as associated strings.

[0070] Specifically, since the server needs to determine the association between the character strings corresponding to the text lines contained in the image, the server can first determine the character strings corresponding to the text lines based on the regions in the image where the text lines are located. The server can input the image corresponding to the regions where the text lines are located into a trained text line recognition model, and obtain the character strings output by the text line recognition model as the character strings corresponding to the text lines contained in the regions where the input text lines are located.

[0071] Common text line recognition models include attention models and connectionist temporal classification (CTC) models. The server can input an image corresponding to the text line region into a trained attention model or CTC model to obtain the model's output string. Currently, using attention models or CTC models to identify text lines contained in images and determine the corresponding strings is a relatively mature technology and will not be further elaborated in this manual.

[0072] Secondly, after determining the character strings corresponding to each text, the server can determine that there is an association between the character strings corresponding to each text line in the designated area based on the position of the area where each text line is located in the image and the position of each designated area in the image. For each designated area, if it is determined that at least two text lines fall into the designated area, then it is determined that there is an association between the character strings corresponding to each text line in the designated area.

[0073] Figure 3 This is a schematic diagram of the area where text lines fall into the same designated area provided in this manual, where the dotted line represents the area where the text line is located, which can be seen to include three text line areas A, B, D, and F, and the solid line represents the designated area, that is, Figure 3 The regions marked as C and E in the figure show that text line regions A and B fall into designated region C, while text line region D does not fall into any designated region, and text line region F falls into designated region E. Since only text line region F falls into designated region E, there is no association between the character strings corresponding to the text lines contained in text line region F and other character strings.

[0074] Of course, the server may first determine that there is an association between the areas where the text lines fall into the same designated area, then determine the character strings corresponding to the areas where the text lines have an association, and determine the association between the character strings based on the association between the areas where the text lines have an association.

[0075] For example, still Figure 3For example, assuming that area A where the text line is located and area B where the text line is located both fall into the designated area C, the server can determine that there is an association relationship between area A where the text line is located and area B where the text line is located, and then determine the string a corresponding to area A where the text line is located and the string b corresponding to area B where the text line is located, and determine that there is an association relationship between string a and string b, which can reduce the resources consumed in identifying strings and improve data processing efficiency.

[0076] S106: Determine the data identifier corresponding to the geographical location when the image was collected, based on the geographical location when the image was collected and the pre-stored correspondence between each data identifier and each geographical location.

[0077] In this specification, since images can only present information within a certain geographic range, the server can update the data within that geographic range based on the string determined by the image. That is, in this specification, the data updated during data processing corresponds to a geographic location (for example, data corresponding to a point of interest). Therefore, after determining the associated string, the server can also determine which data can be updated based on the image obtained in step S100.

[0078] Specifically, the server may first determine the distance between each geographical location and the geographical location at the time of image capture based on the pre-stored geographical locations and the geographical location at the time of image capture. The server may then determine, from the geographical locations, geographical locations whose distance from the geographical location at the time of image capture is less than a predetermined distance. The geographical locations may specifically be the coordinates of pre-stored POIs or pre-stored geographical locations of stores.

[0079] Next, the server may use the data identifiers corresponding to the determined geographic locations that are less than a preset distance from each other as the data identifiers corresponding to the geographic location at the time the image was captured, based on the pre-stored correspondence between the data identifiers and the geographic locations. That is, from the pre-stored geographic locations, the server may determine the geographic locations that are less than a preset distance from the geographic location at the time the image was captured, then determine the corresponding data identifiers based on the pre-stored correspondence between the data identifiers and the geographic locations, and finally use the determined data identifiers as the data identifiers corresponding to the geographic location at the time the image was captured.

[0080] It's important to note that a data identifier refers to the data corresponding to a geographic location, not the identifier of that location. For example, using a POI as an example, this data identifier refers to the data corresponding to the POI, not the identifier used to uniquely identify the POI when it's stored. For example, the data corresponding to a restaurant POI might include the restaurant name, cuisine, contact information, business hours, and so on.

[0081] In addition, since the image acquired by the server should contain the name of the store in the scenario of updating the store information, the server may also use only the store name corresponding to the pre-stored POI as the data identifier.

[0082] Figure 4 This is a schematic diagram of determining the data identifier corresponding to the geographical location when the image was collected, as provided in this specification. Figure 4 The black dots in the middle are POIs on the electronic map. Each POI has several corresponding data identifiers. The white dots are the geographic location when the image was captured. This geographic location has several associated character strings. POIs within the dotted circle are POIs that are less than the preset distance.

[0083] S108: Among the determined data identifiers, determine a data identifier that matches at least one character string as an update identifier.

[0084] In this specification, the character strings with associated relationships determined in step S104 are character strings that can be used for data update, and the specific data that can be updated can be determined based on the data identifier corresponding to the geographical location when the image was collected determined in step S106.

[0085] Specifically, the server can determine the similarity between each data identifier and each string determined in step S104, and determine that the data and string with similarity greater than a preset value match, and use the data identifier that matches at least one string as the update identifier. Figure 4 Taking this as an example, the server can determine the data identifier that matches the string contained in the image from each data identifier. Since the strings determined in step S104 are all associated strings, when the data identifier matches a certain string, it means that other strings associated with the matched string are also related to the data identifier (i.e., the update identifier), and the subsequent steps can update the data accordingly.

[0086] For example, assuming that string a matches data identifier c, and string a is associated with string b, then string b is also associated with data identifier c and can be used to update the data corresponding to the geographic location corresponding to data identifier c. More specifically, assuming that data identifier c is a store name, which corresponds to the geographic location of store X, string a is also the store name, and string b is the contact number. If string a matches data identifier c (i.e., the store name matches), the contact number should be the contact number of store X. In subsequent steps, the data corresponding to store X's geographic location can be updated based on string b.

[0087] S110: updating data corresponding to the update identifier according to other character strings associated with the character string matching the update identifier.

[0088] In this specification, after determining the update identifier, the server may first determine other strings associated with the string based on the string matching the update identifier and the association relationship determined in step S104, and update the data corresponding to the update indication based on the determined other strings.

[0089] In addition, when updating data, the server can first determine a character string that is different from the data corresponding to the update identifier and does not conflict with other character strings that are associated with the character string that matches the update identifier, and then add the determined character string to the data corresponding to the update identifier. Among them, the character string that does not conflict with the data corresponding to the update identifier is a character string whose similarity is lower than a preset threshold. The data corresponding to the update identifier and the character string conflict, which means that the data type is the same but the content is different. For example, the data corresponding to the update identifier includes: telephone, and the content is 123, and the character string is also telephone, but the content is 124, then there is a conflict between the two data. When the recognized data in the image and the pre-stored data correspond to the same geographical location but are different, it is difficult to determine which data is valid. At this time, the server can send the conflicting data to the staff for manual review.

[0090] If the data corresponding to the update identifier does not conflict with the string, the server may update the data corresponding to the update identifier based on the string. If the data corresponding to the update identifier differs from the string, the server may save the determined string as the data corresponding to the update identifier. If the data corresponding to the update identifier and the string are the same, the server may determine that the data corresponding to the updated update identifier has not changed, meaning that there is no new data to save.

[0091] based on Figure 1The data processing method shown, after acquiring an image, first identifies the character strings corresponding to each text line in the image whose area falls within the same designated area of ​​the image as associated character strings. Then, based on pre-stored correspondences between data identifiers and geographic locations, the data identifier corresponding to the geographic location at the time of image acquisition is determined. From the determined data identifiers, a data identifier that matches at least one character string is identified as an update identifier. Finally, based on other character strings associated with the character string matching the update identifier, the data corresponding to the update identifier is updated. By determining the associated relationships among the character strings contained in the acquired image and determining the character string for data update, the data corresponding to the data identifier matching at least one character string is updated, thus realizing a novel data update method that reduces the difficulty of data supplementation.

[0092] In addition, in one or more embodiments provided in this specification, when determining the area where the text line in the image is located in step S102, since the image of the storefront is usually not collected directly on the storefront, there may be a certain angle, resulting in the text line in the image of the storefront showing a perspective effect of being larger near and smaller far away. This results in the minimum circumscribed rectangle of the text line in the image determined by the embodiments of this specification containing more background noise (i.e., non-text area) at the far end of the text line (i.e., the end with smaller text), which may affect the recognition result of the text line in the image, such as Figure 5 shown.

[0093] Figure 5 A schematic diagram of the minimum bounding rectangle (MBR) determined for a non-frontal image. The left side shows the original image, and the center shows the text and non-text line regions segmented by the FCN model. Lighter colors represent text lines, while darker colors represent non-text line regions. The right side shows the minimum bounding rectangle (MBR) determined based on the segmented regions, represented by a dotted line. It can be seen that at the far end of the text line, the MBR includes a significant amount of background. Subsequent recognition based on the image region corresponding to this MBR may reduce the accuracy of character string recognition for the text line.

[0094] Therefore, in this specification, when determining the area where the text line is located, the server can first determine the polygonal area corresponding to at least one text line in the image, and then for each polygonal area, determine the vertical direction of the text line contained in the polygonal area in the image according to a pre-trained angle correction model, and then determine the minimum circumscribed rectangle corresponding to the polygonal area based on the polygonal area, and finally adjust the position and length of the vertical side in the minimum circumscribed rectangle corresponding to the polygonal area according to the determined vertical direction of the text and the polygonal area, and determine the circumscribed quadrilateral area corresponding to the text line based on the adjusted vertical side as the area where the text line is located.

[0095] Specifically, the server may first input the image into a pre-trained FCN model to obtain the image segmentation result output by the FCN model. The image segmentation result may be as follows: Figure 5 The middle image shows the polygons containing text lines and non-text lines obtained by segmenting the image.

[0096] Due to the image acquisition angle, in addition to the possible perspective relationship of larger characters when closer and smaller characters when farther away, the text lines in the image may also have characters that are not on the same horizontal line. Therefore, in order to correct the problem of characters in the text lines not being on the same horizontal line, the server can determine the vertical direction of the text lines contained in each determined polygonal area through a pre-trained angle correction model.

[0097] Specifically, the server can first determine, for each polygonal area, the image containing the polygonal area. The server can convert other polygonal areas in the image into non-text areas to obtain an image containing only the polygonal area, or crop the image vertically and horizontally to obtain an image containing only the polygonal area, while maintaining the vertical and horizontal directions of the cropped image consistent with the vertical and horizontal directions of the image before cropping. Of course, the server can also use other methods to determine the image containing the polygonal area, and this manual does not limit this. By determining the image containing the polygonal area, interference of other polygonal areas in the image with the model output results can be avoided.

[0098] Next, the server may input an image containing the polygonal area into a pre-trained angle correction model to determine the angle between the polygonal area and the horizontal direction of the image. The angle correction model may output a tangent value of the angle, which has a range of values ​​from -1 to 1. The angle between the polygonal area and the horizontal direction of the image may be determined based on the tangent value. A negative tangent value indicates that the font in the polygonal area is tilted to the left in the image, while a negative tangent value indicates that the font is tilted to the right.

[0099] Finally, the server may determine the vertical direction of the characters in the text line contained in the polygonal area in the image according to the determined angle. Figure 6 A schematic diagram of determining the vertical orientation of text provided in an embodiment of this specification. The light-colored area represents the polygonal area of ​​the image containing the Chinese text line, the straight line represents the determined vertical orientation of the text, and the dashed line represents the horizontal inverse of the image. The angle correction model outputs the tangent value of the angle between the straight line and the dashed line, and the server determines the vertical orientation of the text based on this tangent value.

[0100] In addition, in this specification, the angle correction model can specifically be a regression model, such as logistic regression or linear regression. In this specification, the angle correction model can be trained based on pre-prepared training samples. Specifically, the server can obtain images of training samples, typically images containing text lines from a database. The "label" of each training sample is then determined. Specifically, a staff member can annotate the upper left and lower left corners of the text line in the image. The server then determines, for each annotated image, the tangent value of the angle between the line connecting the annotated upper left and lower left corners of the image and the horizontal direction of the image. For example, assuming the upper left corner is P0 and the lower left corner is P3, the server can determine the tangent value as P0(x)-P3(x) / P0(y)-P3(y), where P0(x) and P0(y) are the x-axis and y-axis coordinates of the pixel at point P0 in the image, respectively. Similarly, P3(x) and P3(y) are the x-axis and y-axis coordinates of the pixel at point P3 in the image, respectively. Then, for each labeled image, the image is segmented using the FCN model to obtain an image containing the polygonal area in the image as a training image, and a training sample containing the training image and the tangent value is determined.

[0101] After determining the training sample, the server can minimize the difference between the tangent value output by the angle correction model and the tangent value contained in the training sample as the optimization goal, and adjust the model parameters in the angle correction model until the training end condition is met.

[0102] Furthermore, in this specification, the output of the angle correction model can also be configured as needed, and when training the angle correction model, the model parameters of the angle correction model can be adjusted using corresponding training samples and optimization objectives, which is not a limitation of this specification. For example, if the output is the angle between the polygonal area and the vertical direction of the image, then the corresponding angle value needs to be annotated when determining the training samples, and the angle correction model can also be obtained through a similar training process.

[0103] In this specification, to adjust the minimum bounding rectangle (MBR) corresponding to the polygonal area, thereby reducing background noise in the non-text portion of the image analysis area and improving recognition accuracy, the server may also determine the MBR corresponding to each polygonal area. Of course, since determining the MBR corresponding to a polygonal area is a relatively mature technology, this specification does not elaborate on this process.

[0104] Finally, in this specification, for each polygon, after determining the vertical direction of the text line contained in the polygon and the minimum enclosing rectangle corresponding to the polygon, the server can adjust the position and length of the vertical side of the minimum enclosing rectangle according to the vertical direction of the text to determine the circumscribed quadrilateral area corresponding to the text line contained in the polygon, and based on the obtained circumscribed quadrilateral area, determine the area in the image where the text line corresponding to the character string matching the sample label is located as the area where the text line is located.

[0105] When determining the circumscribed quadrilateral area, the server may first determine the straight line on which the vertical sides of the circumscribed quadrilateral corresponding to the text line lie, based on the determined minimum circumscribed rectangle and the vertical orientation of the text. Specifically, for each vertical side of the minimum circumscribed rectangle, i.e., the left and right vertical sides, the server may determine a straight line passing through the midpoint of each vertical side along the vertical orientation of the text, as the straight line on which the vertical sides of the circumscribed quadrilateral corresponding to the text line lie. In other words, the angles of the vertical sides of the minimum circumscribed rectangle are adjusted to align with the vertical orientation of the text.

[0106] The server can then determine the circumscribed quadrilateral area corresponding to the text line based on the determined straight line where the vertical edge is located, each corner point of the convex hull corresponding to the polygonal area, and the four corners of the minimum circumscribed rectangle. Specifically, the server can first determine each corner point corresponding to the polygonal area as a first-type point, determine the point where each corner of the minimum circumscribed rectangle is located as a second-type point, and then, for each second-type point, determine the line connecting each first-type point and the second-type point, and determine the intersection point of the straight line where each connecting line is located and the straight line where each vertical edge is located, such as Figure 7 shown.

[0107] Figure 7 This is a connection diagram provided in this manual. The black dots are the first type of dots, and the white dots are the second type of dots. Figure 7 The line connecting each first type point and the second type point in the upper left corner of the minimum enclosing rectangle is shown in the figure, that is, the dotted line in the figure, and the adjusted vertical side on the left side of the minimum enclosing rectangle (that is, the straight line where the vertical side of the circumscribed quadrilateral is located, the light-colored straight line in the figure). The server can determine the intersection of each dotted line and the light-colored straight line.

[0108] The server can then determine the corners of the circumscribed quadrilateral corresponding to the text line from each intersection point based on the positional relationship between the second type point and other second type points. That is, adjust the length of the vertical side of the minimum circumscribed rectangle. For example Figure 7The intersection point closest to the image's upper border serves as the vertex of the adjusted minimum bounding rectangle's vertical side, which is the upper left corner of the circumscribed quadrilateral. The intersection point closest to the image's lower border serves as the other vertex of the adjusted minimum bounding rectangle's vertical side, which is the lower left corner of the circumscribed quadrilateral. The distance from this upper left corner to the lower left corner is the length of the adjusted minimum bounding rectangle's vertical side, which also serves as the left vertical side of the circumscribed quadrilateral. Similarly, the server uses the same method to determine the right vertical side of the circumscribed quadrilateral.

[0109] Finally, the server may determine the circumscribed quadrilateral area, that is, the area enclosed by the four corners of the determined circumscribed quadrilateral, based on the points where the corners of the determined circumscribed quadrilateral are located, and use the circumscribed quadrilateral area as the area where the text line is located. Figure 8 Based on Figure 5 The diagram shows a schematic diagram of the circumscribed quadrilateral finally determined by the text lines, wherein the area enclosed by the grey line segments is the circumscribed quadrilateral.

[0110] In addition, in one or more embodiments of the present specification, after the server determines the circumscribed quadrilateral in step S104, since the input of the text line recognition model is usually a rectangular image, in order to facilitate text line recognition, the server may continue to perform image processing on the circumscribed quadrilateral and adjust the circumscribed quadrilateral to a rectangle.

[0111] Specifically, the server may use image processing methods such as stretching, rotating, and distorting to process the circumscribed quadrilateral, adjust the circumscribed quadrilateral into a rectangle, and perform coordinate transformation processing on the text line within the circumscribed quadrilateral to uniformly size the characters in the converted text line within the rectangle, and use the resulting rectangular image as the area containing the text line. Alternatively, the server may directly perform image processing methods such as stretching, rotating, and distorting on the circumscribed quadrilateral to adjust the circumscribed quadrilateral into a rectangle.

[0112] Finally, the region where the text line is located (ie, the converted rectangle) obtained through the above processing is clipped, thereby obtaining the image of the input text line recognition model in step S104.

[0113] The first case can be handled as follows Figure 9 shown. Figure 9 This is a schematic diagram of the process of determining the area where a text line is located provided in an embodiment of this specification, wherein the server can first determine the polygon corresponding to the text line, and then determine the minimum bounding rectangle, and determine the bounding quadrilateral corresponding to the text line through the above process, such as Figure 8 As shown in , the circumscribed quadrilateral is finally adjusted to a rectangle and used as the area where the text line is located. Figure 9 As can be seen from the sample image m, the server can directly perform stretching processing based on the circumscribed quadrilateral to determine the corresponding rectangle. The sample image obtained by the server is Figure 9 The image m in . Among them, it is determined that the circumscribed quadrilateral reduces Figure 8 The problem of more background noise at the far end due to perspective is solved. However, stretching and other processing will not introduce new background noise, but only perform image processing on the original background noise.

[0114] The second treatment situation can be Figure 10 shown. Figure 10 This is a schematic diagram of the process of determining the area where a text line is located provided in an embodiment of this specification. The server can first determine the rectangle corresponding to the circumscribed quadrilateral through stretching processing, and then further perform coordinate change processing on the text line in the circumscribed quadrilateral to adjust the size and position of the smaller characters in the text line to unify the size of each character. The area where the text line is located is determined as follows: Figure 10 As shown in the image n, the originally small characters in the image are magnified due to perspective, further alleviating the problem caused by background noise.

[0115] Furthermore, the server may also directly use the circumscribed quadrilateral corresponding to the text line as the image input to the text line recognition model in step S104. Alternatively, in addition to performing image processing such as stretching, rotating, and twisting on the circumscribed quadrilateral to adjust the circumscribed quadrilateral to a rectangular image, the server may also determine the rectangle circumscribed by the circumscribed quadrilateral and fill the blank area between the circumscribed quadrilateral and the rectangle circumscribed by the circumscribed quadrilateral, for example, by filling with a preset solid color, or determining the average grayscale value of the circumscribed quadrilateral and filling with gray based on the average grayscale value. The rectangle circumscribed by the circumscribed quadrilateral may be the smallest rectangle circumscribed by the circumscribed quadrilateral, or a preset circumscribed rectangle of the same size. This specification does not impose any restrictions and may be set as needed. Furthermore, how to fill the blank area may also be set as needed. For example, a method that introduces less background noise may be selected for filling.

[0116] By transforming the circumscribed quadrilateral into a rectangular image, the generated images of the input text line recognition models can be unified into rectangular images. Of course, the server can also unify the sizes of the images of each input text line recognition model through upsampling or image compression.

[0117] based on Figure 1 The data processing method shown in the embodiment of this specification also provides a structural diagram of a data processing device, such as Figure 11 shown.

[0118] Figure 11 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of this specification, the device comprising:

[0119] An acquisition module 200 acquires an image;

[0120] A first determining module 202 determines an area in the image where a text line is located and at least one designated area in the image;

[0121] The relationship determination module 204 determines the character strings corresponding to the text lines whose regions fall within the same designated region as the character strings having an associated relationship;

[0122] The identifier determination module 206 determines the data identifier corresponding to the geographical location when the image was collected based on the geographical location when the image was collected and the pre-stored correspondence between each data identifier and each geographical location;

[0123] The matching module 208 determines, among the determined data identifiers, a data identifier that matches at least one character string as an update identifier;

[0124] The data processing module 210 updates the data corresponding to the update identifier according to other character strings associated with the character string matching the update identifier.

[0125] Optionally, the acquisition module 200 acquires images collected at the supplier when the delivery vehicle performs a delivery task, and the images at least include the storefront of the supplier.

[0126] Optionally, the identification determination module 206 determines the geographical location when the image is collected based on the coordinates included in the POI of the supplier, or determines the location information of the collection device when the delivery capacity collects the image through the collection device.

[0127] Optionally, the relationship determination module 204 determines the character strings corresponding to each text based on the area where the text lines are located in the image, determines the areas where the text lines are located that fall within the same specified area based on the positions of the designated areas in the image and the positions of the areas where the text lines are located in the image, and determines the association relationship between the character strings corresponding to the areas where the text lines are located that fall within the same specified area.

[0128] Optionally, the identification determination module 206 determines, from the pre-saved geographical locations, geographical locations that are less than a preset distance from the geographical location when the image was collected, and based on the correspondence between the pre-saved data identifiers and the geographical locations, uses the data identifiers corresponding to the determined geographical locations that are less than the preset distance as the data identifiers corresponding to the geographical location when the image was collected.

[0129] Optionally, the data processing module 210 determines the data corresponding to the update identifier based on the pre-saved correspondence between each data and each geographic location, and the geographic location corresponding to the update identifier, and updates the data corresponding to the update identifier based on other character strings that have an associated relationship with the character string matching the update identifier.

[0130] Optionally, the data processing module 210 determines a character string that does not conflict with the data corresponding to the update identifier from other character strings associated with the character string matching the update identifier, and saves the determined character string as the data corresponding to the update identifier.

[0131] The embodiments of this specification also provide a computer-readable storage medium, which stores a computer program. The computer program can be used to execute any one of the above-mentioned data processing methods.

[0132] based on Figure 1 The data processing method shown in the embodiment of this specification also proposes Figure 12 The schematic structure diagram of the electronic device shown in FIG. Figure 12 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for its operations. The processor reads the corresponding computer program from the non-volatile storage into the memory and then runs it to implement any of the above-mentioned data processing methods.

[0133] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0134] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0135] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0136] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0137] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0138] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0142] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0143] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0144] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0145] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0146] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0148] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0149] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible within the scope of the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A data processing method, characterized in that: include: Acquire images captured by the delivery transport at the supplier's location when the delivery transport is performing a delivery task, wherein the images at least include the storefront of the supplier; determining an area in the image where a text line is located and at least one designated area in the image; Determine the character strings corresponding to the text lines whose regions fall within the same designated region as the character strings having an associated relationship; Determining the data identifier corresponding to the geographical location when the image was collected, based on the geographical location when the image was collected and the pre-stored correspondence between the data identifiers and the geographical locations; Among the determined data identifiers, determining a data identifier that matches at least one character string as an update identifier; updating the data corresponding to the update identifier based on other character strings that are associated with the character string that matches the update identifier; The character strings corresponding to the text lines whose regions fall within the same designated region are determined as the character strings having an associated relationship, specifically including: Determining the character strings corresponding to the respective texts according to the regions where the text lines are located in the image; Determining the regions where the text lines are located that fall within the same designated region according to the positions of the designated regions in the image and the positions of the regions where the text lines are located in the image; Determining the association relationship between the character strings corresponding to the regions where the text lines fall within the same designated region; the determining of the geographical location when the image is collected specifically includes: The geographical location when the image is collected is determined based on the coordinates contained in the POI of the supplier; or the location information of the collection device when the distribution capacity collects the image through the collection device is determined.

2. The method according to claim 1, wherein Determining a data identifier corresponding to the geographic location when the image was collected, specifically comprising: From the pre-saved geographical locations, determine the geographical locations that are less than a preset distance from the geographical location when the image was collected; based on the correspondence between the pre-saved data identifiers and the geographical locations, use the data identifiers corresponding to the determined geographical locations that are less than the preset distance as the data identifiers corresponding to the geographical location when the image was collected.

3. The method according to claim 1, wherein Updating the data corresponding to the update identifier based on other character strings that are associated with the character string matching the update identifier, specifically including: Determining the data corresponding to the update identifier based on the pre-stored correspondence between each data and each geographical location and the geographical location corresponding to the update identifier; The data corresponding to the update identifier is updated according to other character strings that are associated with the character string that matches the update identifier.

4. The method according to claim 3, wherein Updating the data corresponding to the update identifier specifically includes: Determining, from other character strings associated with the character string matching the update identifier, a character string that does not conflict with the data corresponding to the update identifier; The determined character string is saved as data corresponding to the update identifier.

5. A data processing device, characterized in that: include: an acquisition module for acquiring images captured by the delivery transport at the supplier's location when the delivery transport is performing a delivery task, wherein the images at least include the storefront of the supplier; A first determining module determines an area in the image where a text line is located and at least one designated area in the image; The relationship determination module determines the character strings corresponding to the text lines whose regions fall within the same designated area as the character strings having an associated relationship, and determines the character strings corresponding to the text lines whose regions fall within the same designated area as the character strings having an associated relationship, specifically including: Determining the character strings corresponding to the respective texts according to the regions where the text lines are located in the image; Determining the regions where the text lines are located that fall within the same designated region according to the positions of the designated regions in the image and the positions of the regions where the text lines are located in the image; Determine the association relationship between the character strings corresponding to the regions where each text line falls within the same specified region; The identifier determination module determines, based on the geographic location at the time the image was collected and the pre-stored correspondence between each data identifier and each geographic location, a data identifier corresponding to the geographic location at the time the image was collected. The determining of the geographic location at the time the image was collected specifically includes: Determine the geographical location when the image is collected based on the coordinates of the POI of the supplier; or determine the location information of the collection device when the delivery vehicle collects the image through the collection device; A matching module, which determines, among the determined data identifiers, a data identifier that matches at least one character string as an update identifier; The data processing module updates the data corresponding to the update identifier according to other character strings that are associated with the character string that matches the update identifier.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images