Column graph information identification method and apparatus

Abstract

The invention discloses a histogram information identification method and device, and the method comprises the steps: obtaining a histogram image of a histogram needing to be identified; performing target detection on the histogram image to obtain detection information of each element detection area on the histogram image; performing semantic segmentation on the histogram image to obtain segmentation information of each target columnar segmentation region on the histogram image; and determining structured data of the histogram based on the detection information of each element detection region and the segmentation information of each target columnar segmentation region. According to the invention, the accuracy of column region identification can be accurately improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for identifying information of a bar graph. Background Art

[0002] With the development of current office software, more and more people use charts in daily study and work to assist in conveying data statistics to enhance the readability of statistical results. Column charts are commonly used in data analysis and can be used for data comparison, trend display, current situation description, etc. The introduction of column charts can greatly enhance the readability of statistical results. Column charts are mainly composed of rectangular column areas, x-axis titles, y-axis titles, legends, and properties of rectangular column areas. Since charts are usually stored in the form of pictures, if you want to migrate a beautiful and adaptive column chart to a predetermined scene, you can only use it as a reference to re-make it. The existing technology uses detection to detect each element in the column chart, and then generates structured data to facilitate the migration and use of the column chart, but the target detection method has poor detection effect on the rectangular column area in the column chart, resulting in a low accuracy rate in information recognition of the column chart.

[0003] That is, the information recognition accuracy of the bar graph in the prior art is low. Summary of the invention

[0004] The embodiments of the present application provide a method and device for identifying information in a bar graph, which can improve the accuracy of information identification in a bar graph.

[0005] In a first aspect, the information identification method of a bar graph provided by the present application comprises:

[0006] Obtain a column chart image of a column chart to be identified;

[0007] Performing target detection on the columnar image to obtain detection information of each element detection area on the columnar image;

[0008] Performing semantic segmentation on the columnar image to obtain segmentation information of each target columnar segmentation area on the columnar image;

[0009] The structured data of the column chart is determined based on the detection information of each of the element detection areas and the segmentation information of each of the target columnar segmentation areas.

[0010] In a second aspect, the information identification device of the column chart provided by the present application includes:

[0011] An acquisition module, used for acquiring a bar graph image of a bar graph to be identified;

[0012] A target detection module, used to perform target detection on the column chart image to obtain detection information of each element detection area on the column chart image;

[0013] A semantic segmentation module, used to perform semantic segmentation on the columnar image to obtain segmentation information of each target columnar segmentation area on the columnar image;

[0014] A determination module is used to determine the structured data of the column chart based on the detection information of each of the element detection areas and the segmentation information of each of the target columnar segmentation areas.

[0015] In a third aspect, the electronic device provided in the present application includes a memory and a processor, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the bar chart information recognition method provided in the present application.

[0016] In a fourth aspect, the computer-readable storage medium provided in the present application stores a plurality of instructions, which are suitable for loading by a processor to implement the steps in the bar chart information recognition method provided in the present application.

[0017] In a fifth aspect, the computer program product provided in the present application includes a computer program or instructions, which, when executed by a processor, implement the steps in the bar chart information identification method provided in the present application.

[0018] In the present application, compared with the related art, a column chart image of a column chart to be identified is obtained; target detection is performed on the column chart image to obtain detection information of each element detection area on the column chart image; semantic segmentation is performed on the column chart image to obtain segmentation information of each target columnar segmentation area on the column chart image; and structured data of the column chart is determined based on the detection information of each element detection area and the segmentation information of each target columnar segmentation area. The present application identifies each element on the column chart image by target detection, and then identifies the rectangular column area on the column chart image by semantic segmentation, and then determines the structured data based on the recognition result. Since target detection has a higher recognition accuracy rate for elements in non-rectangular column areas, and semantic segmentation has a higher recognition accuracy rate for rectangular column areas, the accuracy of column area recognition can be accurately improved by jointly identifying the column chart image by these two methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 It is a scene schematic diagram of the column chart information recognition system provided by the embodiment of the present application;

[0021] Figure 2 It is a flowchart of an embodiment of a method for identifying information of a bar graph provided in an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a bar graph image in one embodiment of a bar graph information recognition method provided by an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of an element detection area on a column chart image in an embodiment of a column chart information recognition method provided by an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of each initial columnar segmentation area in an embodiment of the method for identifying information of a columnar graph provided by an embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of each target columnar segmentation area or annotated boundary box in an embodiment of the columnar information recognition method provided by the embodiment of the present application;

[0026] Figure 7 is a schematic diagram of filling a bounding box in an embodiment of a method for identifying information of a bar graph provided by an embodiment of the present application;

[0027] Figure 8 It is a flowchart of an embodiment of determining structured data of a column chart based on detection information of each element detection area and segmentation information of each target columnar segmentation area in the column chart information recognition method provided in the embodiment of the present application;

[0028] Fig. 9 It is a flowchart of another embodiment of determining structured data of a column chart based on detection information of each element detection area and segmentation information of each target columnar segmentation area in the column chart information recognition method provided in an embodiment of the present application;

[0029] Fig.10 It is a flowchart of another embodiment of determining structured data of a column chart based on detection information of each element detection area and segmentation information of each target column segmentation area in the column chart information recognition method provided in the embodiment of the present application;

[0030] Fig.11 It is a schematic diagram of an embodiment of obtaining color information in each legend element area and color information in each target columnar segmentation area in the columnar chart information recognition method provided in the embodiment of the present application;

[0031] Fig.12 It is a flowchart diagram of another embodiment of the method for identifying information of a bar graph provided in an embodiment of the present application;

[0032] Fig.13 It is a structural schematic diagram of an embodiment of a bar graph information identification device provided in an embodiment of the present application;

[0033] Fig.14 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] It should be noted that the principles of the present application are illustrated by implementing them in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present application and should not be considered as limiting other specific embodiments of the present application that are not described in detail herein.

[0035] In the following description of the present application, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description of the present application, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0038] In order to improve the accuracy of information recognition of a bar graph, an embodiment of the present application provides a bar graph information recognition method, a bar graph information recognition device, an electronic device, a computer-readable storage medium, and a computer program product. The bar graph information recognition method can be executed by the bar graph information recognition device, or by an electronic device integrated with the bar graph information recognition device.

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0040] Please refer to Figure 1 , the present application also provides a column chart information recognition system, such as Figure 1 As shown, the bar graph information recognition system includes an electronic device 100, and the electronic device 100 is integrated with the bar graph information recognition device provided by the present application.

[0041] Among them, the electronic device 100 can be any device equipped with a processor and has processing capabilities, such as mobile electronic devices with processors such as smart phones, tablet computers, PDAs, laptops, smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, industrial equipment, etc.

[0042] In addition, the bar graph information recognition system may further include a memory 200 for storing original data, intermediate data, and result data.

[0043] In the embodiment of the present application, the memory may be a cloud memory. Cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as the storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of storage devices of various types (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0044] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

[0045] The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of independent redundant disk arrays (RAID, Redundant Array of Independent Disks), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0046] It should be noted that Figure 1 The scenario diagram of the bar chart information identification system shown is merely an example. The bar chart information identification system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the bar chart information identification system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0047] It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0048] Please refer to Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of a method for identifying information of a bar graph provided in an embodiment of the present application. Figure 2 As shown, the process of the column chart information recognition method provided by this application is as follows:

[0049] 201. Obtain a bar graph image of a bar graph to be identified.

[0050] The column chart image is an image file containing column chart content. The format of the column chart image can be jpg format, png format or bmp format, etc., which is set according to specific circumstances. The column chart content of the column chart image includes the column area of ​​the column chart and its related text information.

[0051] In the embodiment of the present application, the bar chart file displayed on the screen can be intercepted by a screenshot tool to obtain a bar chart image. The bar chart file opened can also be exported as a bar chart image by bar chart software. The bar chart image can also be downloaded or intercepted from the Internet, or a clear bar chart image can be taken from a display device.

[0052] like Figure 3 and Figure 4As shown, the column chart image includes multiple elements, including the title of the first axis alignment, the value of the first axis alignment, the title of the second axis alignment, the value of the second axis alignment, each column area of ​​the column chart, the legend, and the title of the chart. For example, the first axis alignment is the y-axis, and the second axis alignment is the x-axis. Of course, it is also possible that the second axis alignment is the y-axis and the first axis alignment is the x-axis. Among them, the title of the x-axis is Sales. The values ​​of the x-axis are Jan, Feb, Mar, and Apr. The title of the y-axis is Month, and the values ​​of the y-axis are 0-8000, respectively. The legends are Houston and the corresponding color blocks, Dallas and the corresponding color blocks. The values ​​of the column areas include percentages and values. For example, the values ​​of the column areas corresponding to Houston and Jan are 3k, and the percentage is 22%.

[0053] 202. Perform target detection on the columnar image to obtain detection information of each element detection area on the columnar image.

[0054] The task of object detection is to find all the objects of interest in the image and determine their categories and locations. It is one of the core issues in the field of computer vision. Since various objects have different appearances, shapes and postures, and are interfered by factors such as illumination and occlusion during imaging, object detection has always been the most challenging problem in the field of computer vision. Object detection algorithms based on deep learning are mainly divided into two categories: Two stage and One stage. TowStage: First determine the region, which is called region proposal (RP for short, a pre-selected box that may contain the object to be detected), and then classify the samples through convolutional neural networks. Task flow: feature extraction, determination of RP, classification / localization regression. Common tow stage object detection algorithms include: R-CNN, SPP-Net, Fast R-CNN, Faster R-CNN and R-FCN. One Stage, directly extract features in the network to predict object classification and location. Task flow: feature extraction, classification / localization regression. Common one-stage target detection algorithms include: OverFeat, YOLOv1, YOLOv2, YOLOv3, SSD and RetinaNet.

[0055] In the embodiment of the present application, a pre-trained preset target detection model is used to perform target detection on the column chart image, and each element detection frame on the column chart image is obtained, and the area within the element detection frame is determined as the element detection area. Among them, the preset target detection model can be a model such as YOLOv5, NanoDet, FasterRCNN, etc., which can be selected according to the specific situation, and this application does not limit this.

[0056] By inputting the acquired column chart image into the preset target detection model, a series of element detection frames can be obtained, and the area within the element detection frame is determined as the element detection area. At the same time, the preset target detection model outputs the detection information of each element detection area. The detection information includes position information and text category. The text category can be represented by label, and the position information is represented by point, where point represents the coordinates of the upper left corner and the lower right corner of the rectangular box. Among them, label represents the text category in the element detection frame, etc.

[0057] In the embodiment of the present application, each element detection area includes a first dimensional element area, a second dimensional element area, and a legend element area.

[0058] like Figure 4 As shown, Figure 4 The figure shows the various element detection areas on the bar chart image output by the preset target detection model. For example, the detection information of one element detection area is: {label: x_axis_value, point[(132,227), (154,290)]}, which means that the attribute information of the element detection area is the title of the second alignment axis, that is, the title x_axis_value of the x-axis, and the position information of the element detection area is: the coordinates of the upper left corner and the lower right corner are (132,227) and (154,290) respectively.

[0059] In the embodiment of the present application, multiple columnar image samples are collected, and each element in each columnar image sample is annotated to obtain training data, which includes rectangular annotation boxes on each columnar image sample; the training data is used to train a preset target detection model. The rectangular annotation boxes all have text category and position information, and the form of the rectangular annotation boxes is similar to Figure 4 The element detection area predicted in is the same as Figure 4 The rectangular annotation box in can be expressed as {label: x_axis_value, point[(132,227), (154,290)]}. It should be noted that due to the diversity of column charts, the column area contains vertical or horizontal display situations, and the positions of the corresponding two-dimensional coordinate axes are different. Here, this application defines that in all cases, the vertical coordinate axis is the y-axis, and the corresponding value is y_axis_value; the horizontal coordinate axis is the x-axis, and the corresponding value is x_axis_value.

[0060] 203. Perform semantic segmentation on the columnar image to obtain segmentation information of each target columnar segmentation area on the columnar image.

[0061] Semantic segmentation, literally means letting the computer segment the image based on its semantic meaning. Semantic segmentation segments different objects in the image from the perspective of pixels, labels each pixel in the original image, and classifies each pixel in the image.

[0062] In the embodiment of the present application, a preset semantic segmentation model is used to perform semantic segmentation on the columnar image to obtain each target columnar segmentation area on the columnar image. The preset semantic segmentation model can be an Alexnet model, a GoogLeNet model, etc., which can be selected according to the specific situation. The segmentation information includes the position information of each pixel point of the target columnar segmentation area.

[0063] In a specific embodiment, the columnar image is input into a preset semantic segmentation model to obtain initial columnar segmentation regions on the columnar image, and each initial columnar segmentation region is determined as a corresponding target columnar segmentation region. Figure 5 As shown, Figure 5 The figure shows the schematic diagram of each initial columnar segmentation area on the columnar image output by the preset semantic segmentation model. Figure 5 The boundaries of the initial columnar segmentation regions in have prominent burrs.

[0064] In the embodiment of the present application, a plurality of columnar image samples are collected, and the column area in each columnar image sample is annotated with a bounding box to obtain an annotated bounding box, and the annotated bounding box is converted into a binary mask, which is used as a label mask for a preset semantic segmentation model to obtain a training data set. The preset semantic segmentation model is then trained using the training data set. At this time, the bounding box annotations on the columnar image samples are as follows: Figure 6 As shown in . Of course, the area within the bounding box annotation can also be filled to obtain a filled bounding box, and the filled bounding box can be converted into a binary mask and used as a label mask for the preset semantic segmentation model to obtain a training data set. Then the training data set is used to train the preset semantic segmentation model. At this time, the filled bounding box annotated on the columnar image sample is as follows Figure 7 As shown in Figure 2, both types of masks are used to train the edge semantic segmentation model.

[0065] Since the segmented area obtained by semantic segmentation may have burrs, etc., which may cause the segmented area obtained by semantic segmentation to not match the actual rectangular column area, in order to improve the segmentation accuracy of the target columnar segmented area, in another specific embodiment, semantic segmentation is performed on the columnar image to obtain segmentation information of each target columnar segmented area on the columnar image, which may include:

[0066] (1) Input the column chart image into a preset semantic segmentation model to obtain an initial columnar segmentation area on the column chart image.

[0067] (2) Preprocessing each initial columnar segmentation region to obtain a preprocessed columnar region corresponding to each initial columnar segmentation region.

[0068] The preprocessing includes at least one of connected domain analysis and morphological operation.

[0069] In the embodiment of the present application, a connected domain analysis is performed on the initial columnar segmentation area to obtain a connected domain image; a morphological operation is performed on the connected domain image to remove burrs to obtain pre-processed columnar areas corresponding to each initial columnar segmentation area.

[0070] (3) The circumscribed rectangular areas of the preprocessed columnar areas are respectively determined as the target columnar segmentation areas, and the segmentation information of the target columnar segmentation areas is obtained.

[0071] Specifically, yes Figure 5 The initial columnar segmentation regions of the columnar image are processed to obtain the target columnar segmentation regions of the columnar image. The target columnar segmentation regions of the columnar image are as follows: Figure 6 shown.

[0072] 204. Determine structured data of the column graph based on the detection information of each element detection area and the segmentation information of each target column segmentation area.

[0073] In a specific embodiment, the detection information of each element detection area and the segmentation information of each target columnar segmentation area are aggregated to obtain structured data of a column chart.

[0074] In the embodiment of the present application, the element detection region is a first dimensional element region. One side of each target columnar segmentation region is aligned with a first alignment axis, and the structured data includes first dimensional attribute information of each target columnar segmentation region.

[0075] In an embodiment of the present application, multiple edges of the same length in multiple target columnar segmentation areas are obtained, and the straight line where the multiple edges of the same length are located is determined as the second alignment axis, and the axis perpendicular to the second alignment axis is determined as the first alignment axis. The multiple edges of the same length in multiple target columnar segmentation areas are the wide sides of the target columnar segmentation areas. Specifically, the first alignment axis is the y-axis, the first dimensional element area is the detection box where the title y_axis_value of the first alignment axis is located, the second alignment axis is the x-axis, and the second dimensional element area is the detection box where the title x_axis_value of the second alignment axis is located.

[0076] The present application identifies each element on the column chart image through target detection, then identifies the rectangular column area on the column chart image through semantic segmentation, and then determines the structured data based on the recognition results. Since target detection has a high recognition accuracy for elements in non-rectangular column areas, and semantic segmentation has a high recognition accuracy for rectangular column areas, the accuracy of column area recognition can be accurately improved by jointly identifying the column chart image through these two methods.

[0077] See also Figure 8 In the embodiment of the present application, the structured data of the column chart is determined based on the detection information of each element detection area and the segmentation information of each target column segmentation area, including:

[0078] 301. Determine a first distance parameter between a first first-dimensional element region and a last first-dimensional element region among multiple first-dimensional element regions based on detection information and segmentation information.

[0079] In a specific embodiment, the first distance parameter is the distance between the center coordinates of the first first dimensional element area and the center coordinates of the last first dimensional element area. Specifically, the detection information includes the position information of the element detection area, and the segmentation information includes the position information of the first dimensional element area. The center coordinates of the first first dimensional element area and the center coordinates of the last first dimensional element area are obtained according to the detection information and the segmentation information. The center coordinates of the first first dimensional element area are marked as (center_text_x1, center_text_y1), the center coordinates of the last first dimensional element area are marked as (center_text_x2, center_text_y2), and the first distance parameter is (center_text_y1-center_text_y2).

[0080] 302. Recognize the text in the first first-dimensional element region and the last first-dimensional element region to obtain the numerical value in the first first-dimensional element region and the numerical value in the last first-dimensional element region.

[0081] Specifically, the value in the first first-dimensional element area is value1, and the value in the last first-dimensional element area is value2. Figure 4 As shown, the value value1 in the first first-dimensional element area is 8000, and the value value2 in the last first-dimensional element area is 0.

[0082] 303 . Determine first dimensional attribute information of the target columnar segmentation region based on a difference between a value in the first first dimensional element region and a value in the last first dimensional element region, a first distance parameter, and a height of the target columnar segmentation region.

[0083] Among them, the first dimension attribute information is the height value of the bar graph, such as Figure 3 As shown, the bar graph height value is marked on the bar area, for example, the bar graph height value is 3k.

[0084] In a specific embodiment, the numerical difference between the first first dimensional element region and the last first dimensional element region is (value1-value2). The numerical difference (value1-value2), the first distance parameter (center_text_y1-center_text_y2), the height height of the target columnar segmentation region, and the bar graph height value bar_value satisfy the following formula:

[0085] bar_value=(height / (center_text_y1-center_text_y2))*(value1-value2)

[0086] Among them, bar_value is the height value of the bar chart, (value1-value2) is the difference between the value in the first first dimension element area and the value in the last first dimension element area, (center_text_y1-center_text_y2) is the first distance parameter, and height is the height of the target columnar segmentation area.

[0087] Since the distribution and marking of the bar chart height values ​​on the bar chart image are relatively messy, the conventional target detection accuracy is low. The present application determines the first dimensional attribute information of the target columnar segmentation area through the numerical difference, the first distance parameter and the height of the target columnar segmentation area. Compared with directly detecting the first dimensional attribute information through target detection, the present application has a higher recognition accuracy.

[0088] See also Fig. 9 In a specific embodiment, the element detection area is a second dimensional element area, one side of each second dimensional element area is aligned with the second alignment axis, and the structured data includes the second dimensional attribute information of each target columnar segmentation area. Specifically, the second alignment axis is the x-axis, the second dimensional element area is the detection box where the value x_axis_value of the second alignment axis is located, and the second dimensional attribute information is the value of the second alignment axis. For example, the value x_axis_value of the second alignment axis is Jan. The structured data of the column chart is determined based on the detection information of each element detection area and the segmentation information of each target columnar segmentation area, including:

[0089] 401. Obtain a segmentation region cluster obtained by clustering multiple target columnar segmentation regions based on detection information and segmentation information.

[0090] The segmentation region cluster includes at least two target columnar segmentation regions.

[0091] In a specific embodiment, clustering multiple target columnar segmentation regions may include:

[0092] (1) Obtain a third distance parameter between each target columnar segmentation region on the second alignment axis.

[0093] In a specific embodiment, the center coordinates of the two target columnar segmentation regions are obtained, the center coordinates of the two target columnar segmentation regions are projected on the second alignment axis, and the distance between the projection points of the center coordinates of the two target columnar segmentation regions on the second alignment axis is determined as the third distance parameter. In other embodiments, the third distance parameter may also be represented by other distances.

[0094] (2) Clustering the plurality of target columnar segmentation regions based on the second distance parameter between each target columnar segmentation region to obtain a segmentation region cluster, wherein the third distance parameter between any two adjacent target columnar segmentation regions in the segmentation region cluster is less than a preset distance parameter.

[0095] The preset distance parameter can be one third of the width of the target column segmentation area, which can be set according to the specific situation. Since the column areas under the same legend in the column chart are close to each other, clustering is performed using the third distance parameter between the target column segmentation areas for this feature of the column chart, which can simplify the clustering method and improve the clustering accuracy, thereby improving the recognition accuracy.

[0096] In other embodiments, clustering may also be performed using other methods such as the K-MEANS clustering algorithm, which is not limited here.

[0097] 402. Calculate a second distance parameter between each target columnar segmentation region and a second dimensional element region in the segmentation region cluster.

[0098] The second distance parameter is the distance between the midpoint coordinates of the alignment edge of the target columnar segmentation area and the midpoint coordinates of the alignment edge of the second dimensional element area. The alignment edge of the target columnar segmentation area is the edge of the target columnar segmentation area aligned with the second alignment axis, and the alignment edge of the second dimensional element area is the edge of the second dimensional element area aligned with the second alignment axis. Figure 5As shown, the alignment edge of the target columnar segmentation area is the bottom edge of the target columnar segmentation area, and the alignment edge of the second dimensional element area is the bottom edge of the second dimensional element area. Specifically, the midpoint coordinates of the alignment edge of the target columnar segmentation area are (center_bar_x, center_bar_y), and the midpoint coordinates of the alignment edge of the second dimensional element area are (center_text_x, center_text_y).

[0099] For each segmentation region cluster, a second distance parameter between each target columnar segmentation region and the second dimensional element region in the segmentation region cluster is calculated respectively.

[0100] In a specific embodiment, in order to verify whether the clustering is correct, it is determined whether the number of segmentation region clusters is the same as the number of second dimensional element regions. If the number of segmentation region clusters is the same as the number of second dimensional element regions, it indicates that the number of clusters is correct. Then, it is determined whether the number of target columnar segmentation regions in each segmentation region cluster is the same as the number of legend element regions. If the number of target columnar segmentation regions in each segmentation region cluster is the same as the number of legend element regions, the second distance parameter between each target columnar segmentation region in the segmentation region cluster and the second dimensional element region is calculated.

[0101] 403 . Determine the target columnar segmentation region in the segmentation region cluster whose second distance parameter to the second dimensional element region is the smallest as the target columnar segmentation region corresponding to the second dimensional element region.

[0102] Specifically, for each segmentation region cluster, each target columnar segmentation region in the segmentation region cluster is combined with the second dimensional element region respectively to obtain multiple region combination pairs, each region combination pair is recorded as (center_text, center_bar), and a relationship is established between the region combination pair (center_text, center_bar) with the smallest second distance parameter to obtain the target columnar segmentation region corresponding to each second dimensional element region.

[0103] 404. Assign the text information of the second dimensional element region to the corresponding target columnar segmentation region to obtain the second dimensional attribute information of the target columnar segmentation region.

[0104] In the embodiment of the present application, text recognition is performed on the second dimensional element region to obtain text information of the second dimensional element region, and the text information of the second dimensional element region is assigned to the corresponding target columnar segmentation region to obtain the second dimensional attribute information of the target columnar segmentation region. Figure 3 As shown, the text information of the second dimensional element area is Jan, and the second dimensional attribute information of the target columnar segmentation area is Jan.

[0105] In an embodiment of the present application, the target columnar segmentation area and the corresponding second dimensional element area are determined by distance matching, and then the second dimensional attribute information is assigned to the target columnar segmentation area. The second dimensional attribute information of the target columnar segmentation area can be accurately determined, which can improve the recognition accuracy.

[0106] See also Fig.10 In a specific embodiment, the element detection area is a legend element area, and the structured data includes legend attribute information of each target columnar segmentation area. The legend attribute information includes legend text and legend color information, such as the legend text is Houston and the legend color is red. The structured data of the column chart is determined based on the detection information of each element detection area and the segmentation information of each target columnar segmentation area, including:

[0107] 501. Obtain color information in each legend element area and color information in each target columnar segmentation area.

[0108] like Fig.11 As shown, in the embodiment of the present application, obtaining the color information in each legend element area and the color information in each target columnar segmentation area includes:

[0109] (1) Perform image enhancement on the image L1 in the legend element region to obtain an enhanced image L2.

[0110] (2) Perform gradient calculation on the enhanced image L2 to obtain the gradient image L3.

[0111] (3) Convert the gradient image L3 into an adaptive binary image L4.

[0112] (4) Perform contour detection and contour filling on the binary image L4 to obtain the filled image L5.

[0113] (5) Remove the text in the filled image L5 and remove the noise and corrosion, retain the connected domain with the largest area, and obtain the legend color area L6.

[0114] (6) The bar chart image is cropped according to the circumscribed rectangle of the legend color area L6 to obtain the legend color image L7.

[0115] For example, the coordinates of the bounding rectangle of the legend color area L7 are {m_x1,m_y1,m_x2,m_y2}

[0116] (7) The color representation vector of the legend color image is determined as the color information of the legend color image, and the color representation vector of the target columnar segmentation area is determined as the color information of the legend color image.

[0117] The color representation vector may be three average values ​​of three RGB channels, and the color representation vector is recorded as legend.color.

[0118] 502. Calculate the color matching degree between the color information in each legend element area and the color information in each target columnar segmentation area.

[0119] Specifically, the Euclidean distance between the color representation vector of the legend color image and the color representation vector of the target columnar segmentation region is calculated, and the color matching degree is determined based on the Euclidean distance, wherein the smaller the Euclidean distance, the greater the color matching degree.

[0120] 503. Determine the target columnar segmentation region with the highest color matching degree with the legend element region as the target columnar segmentation region matching the legend element region.

[0121] 504. Assign the text information and color information in the legend element area to the corresponding target columnar segmentation area to obtain the legend attribute information of each target columnar segmentation area.

[0122] The present application matches the colors of the legend element area and the target column segmentation area, and then unifies the colors of the legend element area and the target column segmentation area to avoid color mismatch between the legend element area and the corresponding target column segmentation area, thereby improving recognition accuracy.

[0123] Please refer to Fig.12 , Fig.12 FIG. 1 is a flow chart of another embodiment of the method for identifying information of a bar graph provided in an embodiment of the present application. Fig.12 As shown, the process of the column chart information recognition method provided by this application is as follows:

[0124] 601. Obtain a bar graph image of a bar graph to be identified.

[0125] The column chart image is an image file containing column chart content. The format of the column chart image can be jpg format, png format or bmp format, etc., which is set according to specific circumstances. The column chart content of the column chart image includes the column area of ​​the column chart and its related text information.

[0126] In the embodiment of the present application, the bar chart file displayed on the screen can be intercepted by a screenshot tool to obtain a bar chart image. The bar chart file opened can also be exported as a bar chart image by bar chart software. The bar chart image can also be downloaded or intercepted from the Internet, or a clear bar chart image can be taken from a display device.

[0127] like Figure 3 and Figure 4As shown, the column chart image includes multiple elements, including the title of the first axis alignment, the value of the first axis alignment, the title of the second axis alignment, the value of the second axis alignment, each column area of ​​the column chart, the legend, and the title of the chart. For example, the first axis alignment is the y-axis, and the second axis alignment is the x-axis. Of course, it is also possible that the second axis alignment is the y-axis and the first axis alignment is the x-axis. Among them, the title of the x-axis is Sales. The values ​​of the x-axis are Jan, Feb, Mar, and Apr. The title of the y-axis is Month, and the values ​​of the y-axis are 0-8000, respectively. The legends are Houston and the corresponding color blocks, Dallas and the corresponding color blocks. The values ​​of the column areas include percentages and values. For example, the values ​​of the column areas corresponding to Houston and Jan are 3k, and the percentage is 22%.

[0128] 602. Perform target detection on the column chart image to obtain detection information of each element detection area on the column chart image.

[0129] In the embodiment of the present application, a pre-trained preset target detection model is used to perform target detection on the column chart image, and each element detection frame on the column chart image is obtained, and the area within the element detection frame is determined as the element detection area. Among them, the preset target detection model can be a model such as YOLOv5, NanoDet, FasterRCNN, etc., which can be selected according to the specific situation, and this application does not limit this.

[0130] By inputting the acquired bar graph image into the preset target detection model, a series of element detection frames can be obtained, and the area within the element detection frame is determined as the element detection area. At the same time, the preset target detection model outputs the detection information of each element detection area. The detection information includes position information and attribute information. The attribute information can be represented by label, and the position information can be represented by point, where point represents the coordinates of the upper left corner and the lower right corner of the rectangular box. Among them, the attribute information can include the text category in the element detection frame, etc.

[0131] like Figure 4 As shown, Figure 4 The figure shows the various element detection areas on the bar chart image output by the preset target detection model. For example, the detection information of one element detection area is: {label: x_axis_value, point[(132,227), (154,290)]}, which means that the attribute information of the element detection area is the title of the second alignment axis, that is, the title x_axis_value of the x-axis, and the position information of the element detection area is: the coordinates of the upper left corner and the lower right corner are (132,227) and (154,290) respectively.

[0132] 603. Perform semantic segmentation on the columnar image to obtain segmentation information of each target columnar segmentation area on the columnar image.

[0133] In the embodiment of the present application, semantic segmentation is performed on the columnar image to obtain each target columnar segmentation area on the columnar image, which may include:

[0134] (1) Input the column chart image into a preset semantic segmentation model to obtain an initial columnar segmentation area on the column chart image.

[0135] (2) Preprocessing each initial columnar segmentation region to obtain a preprocessed columnar region corresponding to each initial columnar segmentation region.

[0136] The preprocessing includes at least one of connected domain analysis and morphological operation.

[0137] In the embodiment of the present application, a connected domain analysis is performed on the initial columnar segmentation area to obtain a connected domain image; a morphological operation is performed on the connected domain image to remove burrs to obtain pre-processed columnar areas corresponding to each initial columnar segmentation area.

[0138] (3) The circumscribed rectangular areas of each pre-processed columnar area are respectively determined as each target columnar segmentation area.

[0139] Specifically, yes Figure 5 The initial columnar segmentation regions of the columnar image are processed to obtain the target columnar segmentation regions of the columnar image. The target columnar segmentation regions of the columnar image are as follows: Figure 6 shown.

[0140] 604 . Obtain a third distance parameter between each target columnar segmentation region on the second alignment axis.

[0141] In the embodiment of the present application, the center coordinates of the two target columnar segmentation areas are obtained, the center coordinates of the two target columnar segmentation areas are projected on the second alignment axis, and the distance between the projection points of the center coordinates of the two target columnar segmentation areas on the second alignment axis is determined as the third distance parameter. In other embodiments, the third distance parameter can also be represented by other distances.

[0142] 605 . Cluster the multiple target columnar segmentation regions based on the second distance parameter between the target columnar segmentation regions to obtain a segmentation region cluster.

[0143] The third distance parameter between any two adjacent target columnar segmentation regions in the segmentation region cluster is smaller than the preset distance parameter.

[0144] The preset distance parameter may be one third of the width of the target columnar segmentation region, and may be set according to specific circumstances.

[0145] 606. Determine and obtain a first distance parameter between a first first-dimensional element region and a last first-dimensional element region among the multiple first-dimensional element regions based on the detection information and the segmentation information.

[0146] In a specific embodiment, the first distance parameter is the distance between the center coordinates of the first first-dimensional element region and the center coordinates of the last first-dimensional element region. Specifically, the center coordinates of the first first-dimensional element region are marked as (center_text_x1, center_text_y1), the center coordinates of the last first-dimensional element region are marked as (center_text_x2, center_text_y2), and the first distance parameter is (center_text_y1-center_text_y2).

[0147] 607. Recognize the text in the first first-dimensional element region and the last first-dimensional element region to obtain the numerical value in the first first-dimensional element region and the numerical value in the last first-dimensional element region.

[0148] Specifically, the value in the first first-dimensional element area is value1, and the value in the last first-dimensional element area is value2. Figure 4 As shown, the value value1 in the first first-dimensional element area is 8000, and the value value2 in the last first-dimensional element area is 8000.

[0149] 608. Determine first dimensional attribute information of the target columnar segmentation region based on a difference between a value in the first first dimensional element region and a value in the last first dimensional element region, the first distance parameter, and a height of the target columnar segmentation region.

[0150] Among them, the first dimension attribute information is the height value of the bar graph, such as Figure 3 As shown, the bar graph height value is marked on the bar area, for example, the bar graph height value is 3k.

[0151] In a specific embodiment, the numerical difference between the first first dimensional element region and the last first dimensional element region is (value1-value2). The numerical difference (value1-value2), the first distance parameter (center_text_y1-center_text_y2), the height height of the target columnar segmentation region, and the bar graph height value bar_value satisfy the following formula:

[0152] bar_value=(height / (center_text_y1-center_text_y2))*(value1-value2)

[0153] Among them, bar_value is the height value of the bar chart, (value1-value2) is the difference between the value in the first first dimension element area and the value in the last first dimension element area, (center_text_y1-center_text_y2) is the first distance parameter, and height is the height of the target columnar segmentation area.

[0154] 609. Obtain a segmentation region cluster obtained by clustering multiple target columnar segmentation regions based on the detection information and the segmentation information.

[0155] The segmentation region cluster includes at least two target columnar segmentation regions.

[0156] 610. Calculate a second distance parameter between each target columnar segmentation region and the second dimensional element region in the segmentation region cluster.

[0157] The second distance parameter is the distance between the midpoint coordinates of the alignment edge of the target columnar segmentation area and the midpoint coordinates of the alignment edge of the second dimensional element area. The alignment edge of the target columnar segmentation area is the edge of the target columnar segmentation area aligned with the second alignment axis, and the alignment edge of the second dimensional element area is the edge of the second dimensional element area aligned with the second alignment axis. Figure 5 As shown, the alignment edge of the target columnar segmentation area is the bottom edge of the target columnar segmentation area, and the alignment edge of the second dimensional element area is the bottom edge of the second dimensional element area. Specifically, the midpoint coordinates of the alignment edge of the target columnar segmentation area are (center_bar_x, center_bar_y), and the midpoint coordinates of the alignment edge of the second dimensional element area are (center_text_x, center_text_y).

[0158] For each segmentation region cluster, a second distance parameter between each target columnar segmentation region and the second dimensional element region in the segmentation region cluster is calculated respectively.

[0159] In a specific embodiment, in order to verify whether the clustering is correct, it is determined whether the number of segmentation region clusters is the same as the number of second dimensional element regions. If the number of segmentation region clusters is the same as the number of second dimensional element regions, it indicates that the number of clusters is correct. Then, it is determined whether the number of target columnar segmentation regions in each segmentation region cluster is the same as the number of legend element regions. If the number of target columnar segmentation regions in each segmentation region cluster is the same as the number of legend element regions, the second distance parameter between each target columnar segmentation region in the segmentation region cluster and the second dimensional element region is calculated.

[0160] 611. Determine a target columnar segmentation region in the segmentation region cluster whose second distance parameter to the second dimensional element region is the smallest as the target columnar segmentation region corresponding to the second dimensional element region.

[0161] Specifically, for each segmentation region cluster, each target columnar segmentation region in the segmentation region cluster is combined with the second dimensional element region respectively to obtain multiple region combination pairs, each region combination pair is recorded as (center_text, center_bar), and a relationship is established between the region combination pair (center_text, center_bar) with the smallest second distance parameter to obtain the target columnar segmentation region corresponding to each second dimensional element region.

[0162] 612. Assign the text information of the second dimensional element region to the corresponding target columnar segmentation region to obtain the second dimensional attribute information of the target columnar segmentation region.

[0163] In the embodiment of the present application, text recognition is performed on the second dimensional element region to obtain text information of the second dimensional element region, and the text information of the second dimensional element region is assigned to the corresponding target columnar segmentation region to obtain the second dimensional attribute information of the target columnar segmentation region. Figure 3 As shown, the text information of the second dimensional element area is Jan, and the second dimensional attribute information of the target columnar segmentation area is Jan.

[0164] 613. Obtain color information in each legend element area and color information in each target column segmentation area.

[0165] 614. Calculate the color matching degree between the color information in each legend element area and the color information in each target columnar segmentation area.

[0166] Specifically, the Euclidean distance between the color representation vector of the legend color image and the color representation vector of the target columnar segmentation region is calculated, and the color matching degree is determined based on the Euclidean distance, wherein the smaller the Euclidean distance, the greater the color matching degree.

[0167] 615. Determine the target column segmentation region with the highest color matching degree with the legend element region as the target column segmentation region matching the legend element region.

[0168] 616. Assign the text information and color information in the legend element area to the corresponding target columnar segmentation area to obtain the legend attribute information of each target columnar segmentation area.

[0169] 617 . Summarize the segmentation information of each target columnar segmentation area, the first dimension attribute information of each target columnar segmentation area, the second dimension attribute information of each target columnar segmentation area, and the legend attribute information of each target columnar segmentation area to obtain structured data of the column chart.

[0170] In a specific embodiment, the structured data of the target columnar segmentation region is determined as structured data of a plurality of column graphs. The structured data of a target columnar segmentation region is as follows:

[0171]

[0172] Among them, the id of the target columnar segmentation area is 1, the second dimension attribute information bar_value is 2879, the first dimension attribute information bar_legend is Houston, and the color information in the legend attribute information is bar_color(185,114,74).

[0173] In a specific embodiment, the structured data of each segmentation region cluster is determined as structured data of multiple columnar graphs. The structured data of a segmentation region cluster includes structured data of four target columnar segmentation regions. The IDs of the four target columnar segmentation regions are 1, 2, 3, and 4, respectively. The structured data of a segmentation region cluster is as follows:

[0174]

[0175]

[0176] In order to facilitate better implementation of the bar graph information recognition method provided in the embodiment of the present application, the embodiment of the present application also provides a bar graph information recognition device based on the above bar graph information recognition method. The meanings of the terms are the same as those in the above bar graph information recognition method. For specific implementation details, please refer to the description in the above method embodiment.

[0177] Please refer to Fig.13 , Fig.13 The schematic diagram of the structure of the information identification device of the column chart provided in the embodiment of the present application, the information identification device of the column chart may include an acquisition module 701, a target detection module 702, a semantic segmentation module 703, and a determination module 704, wherein:

[0178] An acquisition module, used for acquiring a bar graph image of a bar graph to be identified;

[0179] A target detection module, used to perform target detection on the column chart image to obtain detection information of each element detection area on the column chart image;

[0180] A semantic segmentation module, used to perform semantic segmentation on the columnar image to obtain segmentation information of each target columnar segmentation area on the columnar image;

[0181] A determination module is used to determine the structured data of the column chart based on the detection information of each of the element detection areas and the segmentation information of each of the target columnar segmentation areas.

[0182] The specific implementation of each of the above modules can be found in the previous embodiments and will not be described in detail here.

[0183] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the processor is used to execute the steps in the bar chart information identification method provided in this embodiment by calling a computer program stored in the memory.

[0184] Please refer to Fig.14 , Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0185] The electronic device may include components such as a processor 101 with one or more processing cores, a memory 102 with one or more computer-readable storage media, a power supply 103, and an input unit 104. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0186] The processor 101 is the control center of the electronic device, which uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 102, and calling data stored in the memory 102. Optionally, the processor 101 may include one or more processing cores; optionally, the processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 101.

[0187] The memory 102 can be used to store software programs and modules. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102. The memory 102 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 102 may also include a memory controller to provide the processor 101 with access to the memory 102.

[0188] The electronic device also includes a power supply 103 for supplying power to each component. Optionally, the power supply 103 can be logically connected to the processor 101 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 103 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0189] The electronic device may further include an input unit 104, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0190] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically in this embodiment, the processor 101 in the electronic device will load the executable code corresponding to one or more computer programs into the memory 102 according to the following instructions, and the processor 101 will execute the steps in the information recognition method of the bar chart provided in this application, such as:

[0191] Obtain a column chart image of a column chart that needs to be identified; perform target detection on the column chart image to obtain detection information of each element detection area on the column chart image; perform semantic segmentation on the column chart image to obtain segmentation information of each target columnar segmentation area on the column chart image; determine the structured data of the column chart based on the detection information of each element detection area and the segmentation information of each target columnar segmentation area.

[0192] It should be noted that the electronic device provided in the embodiment of the present application and the information recognition method of the bar graph in the above embodiment belong to the same concept, and its specific implementation process is detailed in the above related embodiments and will not be repeated here.

[0193] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program stored therein is executed on a processor of an electronic device provided in an embodiment of the present application, the processor of the electronic device executes the steps in the method for identifying information of a bar graph provided in the present application. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0194] The present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes various optional implementations of the above-mentioned column chart information recognition method.

[0195] The above is a detailed introduction to the information identification method and device of a bar graph provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0196] It should be noted that when the above embodiments of the present application are applied to specific products or technologies, the relevant data of the user is involved, and the user's permission or consent is required, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A method for identifying information of a column chart, characterized in that: include: Obtain a column chart image of a column chart to be identified; Performing target detection on the columnar image to obtain detection information of each element detection area on the columnar image; Performing semantic segmentation on the columnar image to obtain segmentation information of each target columnar segmentation area on the columnar image; The structured data of the column chart is determined based on the detection information of each of the element detection areas and the segmentation information of each of the target columnar segmentation areas.

2. The method for identifying information of a column chart according to claim 1, characterized in that: The performing semantic segmentation on the column chart image to obtain segmentation information of each target columnar segmentation area on the column chart image includes: Inputting the column chart image into a preset semantic segmentation model to obtain an initial columnar segmentation area on the column chart image; Preprocessing each of the initial columnar segmentation regions to obtain a preprocessed columnar region corresponding to each of the initial columnar segmentation regions, wherein the preprocessing includes at least one of a connected domain analysis and a morphological operation; The circumscribed rectangular areas of the pre-processed columnar areas are respectively determined as the target columnar segmentation areas, and the segmentation information of the target columnar segmentation areas on the columnar image is obtained.

3. The method for identifying information of a column chart according to claim 1, characterized in that: The element detection region is a first dimensional element region, one side of each of the target columnar segmentation regions is aligned with a first alignment axis, the structured data includes first dimensional attribute information of each of the target columnar segmentation regions, and the structured data of the column chart is determined based on the detection information of each of the element detection regions and the segmentation information of each of the target columnar segmentation regions, including: Determine a first distance parameter between a first first-dimensional element region and a last first-dimensional element region in a plurality of first-dimensional element regions based on the detection information and the segmentation information; Recognize the text in the first element region of the first dimension and the last element region of the first dimension to obtain the value in the first element region of the first dimension and the value in the last element region of the first dimension; The first dimensional attribute information of the target columnar segmentation region is determined based on the numerical difference between the first element region of the first dimension and the last element region of the first dimension, the first distance parameter, and the height of the target columnar segmentation region.

4. The method for identifying information of a column chart according to claim 1, characterized in that: The element detection area is a second dimensional element area, one side of each of the second dimensional element areas is aligned with a second alignment axis, and the structured data includes second dimensional attribute information of each target columnar segmentation area; the structured data of the column chart is determined based on the detection information of each of the element detection areas and the segmentation information of each of the target columnar segmentation areas, including: Acquire a segmented region cluster obtained by clustering the plurality of target columnar segmented regions based on the detection information and the segmentation information, wherein the segmented region cluster includes at least two target columnar segmented regions; Calculating a second distance parameter between each of the target columnar segmented regions and the second dimensional element region in the segmented region cluster; Determine the target columnar segmentation region in the segmentation region cluster whose second distance parameter to the second dimensional element region is the smallest as the target columnar segmentation region corresponding to the second dimensional element region; The text information of the second dimensional element region is assigned to the corresponding target columnar segmentation region to obtain the second dimensional attribute information of the target columnar segmentation region.

5. The method for identifying information of a column chart according to claim 4, characterized in that: The information identification method of the column chart includes: Acquire a third distance parameter between each of the target columnar segmentation regions on the second alignment axis; A plurality of the target columnar segmentation regions are clustered based on the third distance parameter between each of the target columnar segmentation regions to obtain a segmentation region cluster, wherein the third distance parameter between any two adjacent target columnar segmentation regions in the segmentation region cluster is less than a preset distance parameter.

6. The method for identifying information of a column chart according to claim 1, characterized in that: The element detection area is a legend element area, and the structured data includes legend attribute information of each target columnar segmentation area; the structured data of the column chart is determined based on the detection information of each element detection area and the segmentation information of each target columnar segmentation area, including: Acquire color information in each of the legend element regions and color information in each of the target columnar segmentation regions; Calculate the color matching degree between the color information in each of the legend element regions and the color information in each of the target columnar segmentation regions; Determine the target columnar segmentation region having the highest color matching degree with the legend element region as the target columnar segmentation region corresponding to the legend element region; The legend attribute information in the legend element area is assigned to the corresponding target columnar segmentation area to obtain the legend attribute information of each target columnar segmentation area.

7. A column chart information recognition device, characterized in that: include: An acquisition module, used for acquiring a bar graph image of a bar graph to be identified; A target detection module, used to perform target detection on the column chart image to obtain detection information of each element detection area on the column chart image; A semantic segmentation module, used to perform semantic segmentation on the columnar image to obtain segmentation information of each target columnar segmentation area on the columnar image; A determination module is used to determine the structured data of the column chart based on the detection information of each of the element detection areas and the segmentation information of each of the target columnar segmentation areas.

8. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the steps in the method for identifying information of a bar graph as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the method for identifying information of a bar graph according to any one of claims 1 to 6.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps in the method for identifying information of a bar graph described in any one of claims 1 to 6 are implemented.

Images