A method for acquiring and comparing picture and chart data

Abstract

The present invention relates to a method for acquiring and comparing picture and chart data, and to the technical field of picture and chart comparison. In the analysis of picture and chart, based on the first semantic information including the chart type, drawing area and pixel coordinates of key points and the text recognition result of the pixel coordinates of the text and the area to which the text belongs, when mapping the pixel coordinates to the data coordinates, the coordinate axis scale is extracted based on the clustering algorithm to obtain the densest clusters in the X-axis direction and the Y-axis direction respectively; it is judged whether the number of digital areas in the densest clusters in the X-axis direction and the Y-axis direction reaches a preset number threshold, if it reaches, the pixel coordinates are mapped to the data coordinates based on the constructed quantile regression model; if it does not reach, the pixel coordinates are mapped to the data coordinates based on the constructed affine transformation model. The coordinate scale can be more accurately positioned, the influence of OCR recognition noise is reduced, and the accuracy and robustness of the coordinate mapping method are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of picture and chart comparison, and in particular to a picture and chart data acquisition method and a comparison method. Background Art

[0002] In many fields, especially in scientific research and medical fields, chart comparison is a commonly used data analysis method. However, in daily work, a large number of picture charts exist in the form of pictures, which makes it difficult to conduct comparative analysis.

[0003] Traditional image and chart comparison methods treat charts as general images and use image processing to compare pixel values. However, this method cannot accurately capture the differences in the semantic level of data in the charts. Representative software of this type of method include WinMerge and Difflmg‌.

[0004] Although the chart parsing technology based on machine learning can realize the conversion of chart images into data, it is easily affected by OCR noise points, so it is impossible to achieve complete accuracy in the process of mapping pixel coordinates to data coordinates. This makes it difficult to align the coordinates and data volume of the parsed data of two charts, and requires a lot of manual operations or coding work to achieve data alignment and comparison. Summary of the invention

[0005] The technical problem to be solved by the present application is to provide a method for acquiring and comparing picture and chart data, which has the characteristics of being able to more accurately locate coordinate scales, reduce the impact of OCR recognition noise, and improve the accuracy and robustness of the coordinate mapping method.

[0006] In a first aspect, an embodiment provides a method for acquiring picture chart data, comprising:

[0007] Collecting chart images; the chart images include images of line graphs and / or images of histograms;

[0008] Performing chart parsing on the chart image to obtain first semantic information; the first semantic information includes chart type, drawing area and key point pixel coordinates;

[0009] Performing text recognition on the chart image to obtain a text recognition result; the text recognition result includes the text and the pixel coordinates of the area to which the text belongs;

[0010] Mapping pixel coordinates to data coordinates in combination with the first semantic information and the text recognition result to obtain chart data;

[0011] The mapping of pixel coordinates to data coordinates by combining the first semantic information and the text recognition result to obtain chart data includes:

[0012] Based on the clustering algorithm, the coordinate axis scales are extracted to obtain the densest cluster in the X-axis direction and the densest cluster in the Y-axis direction respectively;

[0013] It is determined whether the number of digital areas in the densest cluster in the X-axis direction and the densest cluster in the Y-axis direction reaches the preset threshold value. If so, the pixel coordinates are mapped to the data coordinates based on the constructed quantile regression model to obtain the corresponding chart data; if not, the pixel coordinates are mapped to the data coordinates based on the constructed affine transformation model to obtain the corresponding chart data.

[0014] In one embodiment, the extraction of coordinate axis scales based on a clustering algorithm to obtain the densest cluster in the X-axis direction and the densest cluster in the Y-axis direction respectively includes:

[0015] Filter the regions containing numbers in the text recognition results to obtain a region set as a first set;

[0016] Filter the digital regions in the first set that intersect with the drawing region to form a new digital region set, and use the new digital region set as the second set;

[0017] Based on the second set, a clustering algorithm is implemented to obtain the most dense cluster in the X-axis direction and the most dense cluster in the Y-axis direction.

[0018] In a second aspect, an embodiment provides a method for comparing pictures and charts, and compares the chart data obtained based on the picture and chart data acquisition method described in any one of the above embodiments; the two picture charts compared include: a histogram picture chart and a line chart picture chart, or two histogram picture charts, or two line chart picture charts.

[0019] The beneficial effects of the present invention are:

[0020] Since in the analysis of the picture chart, based on the first semantic information including the chart type, the drawing area and the pixel coordinates of the key points and the text recognition result of the pixel coordinates of the text and the area to which the text belongs, when the pixel coordinates are mapped to the data coordinates, the coordinate axis scales are extracted based on the clustering algorithm to obtain the densest cluster in the X-axis direction and the densest cluster in the Y-axis direction respectively; it is judged whether the number of digital areas in the densest cluster in the X-axis direction and the densest cluster in the Y-axis direction reaches a preset quantity threshold, if so, the pixel coordinates are mapped to the data coordinates based on the constructed quantile regression model to obtain the corresponding chart data; if not, the pixel coordinates are mapped to the data coordinates based on the constructed affine transformation model to obtain the corresponding chart data; this makes it possible to more accurately locate the coordinate scale, reduce the influence of OCR recognition noise, and improve the accuracy and robustness of the coordinate mapping method. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram of a method for obtaining picture and chart data according to an embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of a chart image input by a user in one embodiment of the present application;

[0023] Figure 3 This is an embodiment of the present application based on Figure 2 Schematic diagram of the drawing area and the area to which the key points belong obtained by chart analysis;

[0024] Figure 4 This is an embodiment of the present application based on Figure 2 The schematic diagram is used to perform text recognition to obtain a schematic diagram of the area to which the text belongs;

[0025] Figure 5 This application Figure 1 A method flow diagram of an embodiment of step S40;

[0026] Figure 6 This application Figure 5 A method flow diagram of an embodiment of step S401;

[0027] Figure 7 is a comparison chart of analytical results obtained based on linear regression and quantile regression respectively in one embodiment of the present application;

[0028] Figure 8 This is a flow chart of a method for performing difference point comparison based on a trend comparison method according to an embodiment of the present application;

[0029] Fig. 9 It is a comparison diagram of the fitting effects of the adaptive local linear regression according to an embodiment of the present application and the existing spline fitting and local linear regression.

[0030] In the diagram: 01, drawing area, 02, key point, 03, area where the text belongs. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0032] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0033] The serial numbers assigned to the components in this article, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.

[0034] To facilitate the description of the inventive concept of the present application, the image chart analysis technology is briefly described below.

[0035] The current image and chart analysis relies on OCR technology. The applicant found in the research that although this technology can realize the conversion of chart images to data, when using OCR technology to identify data in images and charts, it is easily affected by other elements in the image and the quality of the image. There is a problem of being unable to accurately identify and divide densely appearing numbers in the chart, which will generate noise points. Therefore, it is impossible to achieve complete accuracy in the process of mapping pixel coordinates to data coordinates.

[0036] In view of this, an embodiment of the present application provides a method for acquiring and comparing picture chart data. In the analysis of the picture chart, based on the first semantic information including the chart type, drawing area and pixel coordinates of key points and the text recognition result including the pixel coordinates of the text and the area to which the text belongs, when mapping the pixel coordinates to the data coordinates, the coordinate axis scale is extracted based on the clustering algorithm to obtain the densest cluster in the X-axis direction and the densest cluster in the Y-axis direction respectively; it is judged whether the number of digital areas in the densest cluster in the X-axis direction and the densest cluster in the Y-axis direction reaches the preset number threshold, if it reaches, the pixel coordinates are mapped to the data coordinates based on the constructed quantile regression model to obtain the corresponding chart data; if it does not reach, the pixel coordinates are mapped to the data coordinates based on the constructed affine transformation model to obtain the corresponding chart data. The coordinate scale can be more accurately located, the influence of OCR recognition noise is reduced, and the accuracy and robustness of the coordinate mapping method are improved.

[0037] In one embodiment of the present application, a method for obtaining image chart data is provided. The method is applicable to obtaining chart data in line chart images and also to obtaining chart data in histogram images. Figure 1 , the method comprising:

[0038] Step S10, collecting chart images. The collected chart images can be images of line graphs or images of histograms.

[0039] Step S20, parsing the chart image to obtain first semantic information, wherein the first semantic information includes chart type, drawing area and key point pixel coordinates.

[0040] In one embodiment, please refer to Figure 2 and Figure 3 According to the chart image input by the user, the chart is parsed based on the ChartOCR model to obtain the first semantic information including at least the chart type, the drawing area 01 and the pixel coordinates of the key point 02.

[0041] In one embodiment, please refer to Figure 3 , the results of drawing area 01 and key point 02 are both quadrilateral areas, and based on the pixel coordinates of the four corner vertices of the quadrilateral area, the four corner vertex coordinates representing drawing area 01 and the four corner vertex coordinates representing key point 02 can be obtained. For any four corner vertex coordinate representation, the pixel coordinates of the upper left corner vertex, the upper right corner vertex, the lower right corner vertex and the lower left corner vertex are included in sequence.

[0042] Those skilled in the art can understand that, based on the content object corresponding to the region, if the content object is tilted, the quadrilateral region may not be a rectangle. Therefore, the quadrilateral region may be a rectangle or a parallelogram.

[0043] In one embodiment, when there are more than two broken lines or more than two histograms in the parsed object, the first semantic information also includes specific type grouping information. The specific type grouping information is the result of the ChartOCR model performing data transformation on the key points according to the chart type and the preset rules, for example, the key points in the line chart need to be grouped according to the corresponding broken lines to obtain grouping information, and the key points in the histogram need to be grouped according to the corresponding rectangles to obtain grouping information.

[0044] Step S30, performing text recognition on the chart image to obtain a text recognition result, wherein the text recognition result includes the text and the pixel coordinates of the area to which the text belongs.

[0045] In one embodiment, please refer to Figure 4 , text recognition can be performed based on the PaddleOCR model to obtain the pixel coordinates of the text and the area 03 to which the text belongs.

[0046] In one embodiment, the area 03 to which the text belongs is composed of a quadrilateral, and the pixel coordinates of the area are represented by the coordinates of the four corner vertices of the quadrilateral, including the pixel coordinates of the upper left vertex, the upper right vertex, the lower right vertex and the lower left vertex in sequence.

[0047] Step S40 , mapping pixel coordinates to data coordinates is performed in combination with the first semantic information and the text recognition result to obtain chart data.

[0048] The coordinate mapping method can adopt the existing technical method. In one embodiment of the present application, a new method for mapping pixel coordinates to data coordinates is provided. This method can more accurately locate the coordinate scale, reduce the noise impact of OCR recognition, and make the recognition result more robust.

[0049] Please refer to Figure 5 , step S40 may include:

[0050] Step S401 , extracting coordinate axis scales based on a clustering algorithm, and obtaining the most dense cluster in the X-axis direction and the most dense cluster in the Y-axis direction respectively.

[0051] In one embodiment, the coordinate axis scale is extracted based on the MeanShift clustering algorithm.

[0052] Regarding the extraction of coordinate axis scales, the current methods usually have an assumption that the coordinate axis scales are located at the edge of the drawing area. Therefore, the coordinate axis scales can be identified by screening the points at the edge of the drawing area. However, the applicant found in the research that this identification method is easily interfered by legends and other text information. In addition, when the coordinate axis is not at the edge of the drawing area, this method is no longer applicable.

[0053] In view of this, in the embodiments of the present application, a more reasonable assumption is based on that on the same coordinate axis, the scale positions are roughly distributed in parallel. Therefore, the coordinate axis scales are concentrated in the area with the densest pixel positions in the OCR recognition results, but because the OCR recognition results may also contain other elements of the picture, this poses certain difficulties in accurately locating the coordinate scales. Therefore, the embodiments of the present application propose to locate the area based on a clustering algorithm to solve this technical difficulty. The MeanShift clustering algorithm is a density-based clustering algorithm. Its core principle is to move the cluster center in the direction of the densest data in the neighborhood during the iteration process. Therefore, it is more suitable for finding the densest cluster. In addition, the MeanShift clustering algorithm does not need to pre-set the number of clusters, which simplifies the parameter selection process.

[0054] Therefore, in one embodiment, please refer to Figure 6 , step S401 may include:

[0055] Step S4011, filtering the regions containing numbers in the text recognition results, and obtaining a region set as a first set.

[0056] In one embodiment, the first set can be expressed as:

[0057] ;

[0058] Among them, C represents the first set, Indicates the first set m element, namely m digital area, M Indicates the number of digital regions, 1≤ m ≤ M .

[0059] For any digital area , its pixel coordinates can be expressed as:

[0060] ;

[0061] in, Indicates the digital area The pixel coordinates of the upper left corner of Indicates the digital area The pixel coordinates of the upper right corner of Indicates the digital area The pixel coordinates of the lower right vertex, Indicates the digital area The pixel coordinates of the lower left corner of the vertex.

[0062] Step S4012: Filter the digital regions in the first set that intersect with the drawing region to form a new digital region set, and use the new digital region set as the second set.

[0063] In one embodiment, the drawing area can be recorded as R , then the drawing area R The pixel coordinates can be expressed as:

[0064] ;

[0065] in, Indicates the drawing area R The pixel coordinates of the upper left corner of Represents the pixel coordinates of the upper right corner of the drawing area R, Represents the pixel coordinates of the lower right corner of the drawing area R. Indicates the drawing area R The pixel coordinates of the lower left corner of the vertex.

[0066] In one embodiment, screening the digital areas in the first set that intersect with the drawing area may include: for any digital area in the first set, calculating the intersection area between the any digital area and the drawing area, and if the intersection area is greater than 0, it means that there is an intersection.

[0067] In one embodiment, when calculating the intersection area, in order to improve the operation efficiency, the area is approximated as a rectangle, and the area of ​​the intersection area is It can be expressed as:

[0068] ;

[0069] in, A is the intermediate variable, .

[0070] when When >0, it is considered that there is an intersection area.

[0071] Step S4013: Based on the second set, a clustering algorithm is implemented to obtain the most dense cluster in the X-axis direction and the most dense cluster in the Y-axis direction.

[0072] In one embodiment, for the X-axis, the average of the vertical coordinates of the four corner vertices of each digital region in the second set is taken to obtain a third set; based on the third set, a clustering algorithm is implemented to obtain the most dense cluster in the X-axis direction. For the Y-axis, the average of the horizontal coordinates of the four corner vertices of each digital region in the second set is taken to obtain a fourth set; based on the fourth set, a clustering algorithm is implemented to obtain the most dense cluster in the Y-axis direction.

[0073] Step S402, determining whether the number regions included in the most dense cluster in the X-axis direction have reached a preset quantity threshold, if yes, proceeding to step S404, if no, proceeding to step S405.

[0074] In one embodiment, the preset quantity threshold may be 2, 3 or 4, and the specific value may be set based on demand.

[0075] In one embodiment, since the clustering algorithm is implemented based on the third set, the most dense cluster in the X-axis direction is obtained. Therefore, each mean in the most dense cluster in the X-axis direction represents a digital area, that is, it is determined whether the number of means included in the most dense cluster in the X-axis direction reaches a preset number threshold, if yes, then proceed to step S404, if no, then proceed to step S405.

[0076] Step S403, determining whether the number regions included in the most dense cluster in the Y-axis direction reaches a preset quantity threshold, if yes, proceeding to step S404, if no, proceeding to step S405.

[0077] In one embodiment, since the clustering algorithm is implemented based on the third set, the most dense cluster in the Y-axis direction is obtained. Therefore, each mean in the most dense cluster in the Y-axis direction represents a digital area, that is, it is determined whether the number of means included in the most dense cluster in the Y-axis direction reaches a preset number threshold, if yes, then proceed to step S404, if no, then proceed to step S405.

[0078] Step S404, mapping pixel coordinates to data coordinates is performed based on the constructed quantile regression model to obtain corresponding chart data.

[0079] The change from the coordinate axis pixel position to the scale is obviously linear. Compared with general linear changes (such as linear regression and affine transformation), quantile regression is more robust and less susceptible to OCR recognition noise. Please refer to Figure 7 When OCR mistakenly identifies the coordinate axis scale 10.3 as 13, the traditional affine mapping or linear regression model will deviate from the correct trend, resulting in inaccurate numerical prediction, while quantile regression is not affected. Therefore, based on the analysis results obtained in the above embodiment, a more accurate picture chart comparison result can be obtained.

[0080] Those skilled in the art can understand that Figure 7 In the figure, the data on the X-axis is the independent variable, and the data on the Y-axis is the dependent variable. This is only an exemplary description of the embodiment of the present application, and can represent a fixed data relationship in any field, and does not have a special meaning.

[0081] In one embodiment, the quantile regression model includes a quantile regression model of the X-axis and a quantile regression model of the Y-axis, and the construction method includes: collecting chart data of known coordinate scale data, and generating a picture from the chart data. Taking the pixel position of the coordinate scale in the picture as the independent variable data set, taking the scale value as the dependent variable data set, training the quantile regression model, and obtaining the constructed quantile regression model. Specifically, it includes:

[0082] The mean of the horizontal coordinates of the pixel coordinates of the four corner vertices of the quadrilateral area where each coordinate scale pixel position in the image is located is taken as the first mean, the first mean is used as the independent variable, and the scale value corresponding to the first mean is used as the dependent variable to train the quantile regression model of the X-axis.

[0083] The mean of the ordinates of the pixel coordinates of the four corner vertices of the quadrilateral area where each coordinate scale pixel position in the image is located is taken as the second mean, the second mean is used as the independent variable, and the scale value corresponding to the second mean is used as the dependent variable to train the quantile regression model of the Y axis.

[0084] For chart data with known coordinate scale data, the coordinate scale data information in the chart data can be read. Based on the one-to-one correspondence between the independent variable data and the dependent variable data in the known data, the quantile regression model can be trained to obtain a constructed quantile regression model.

[0085] In one embodiment, the parameter q (quantile) of the quantile regression model may be set to 50%.

[0086] Step S405 , mapping pixel coordinates to data coordinates is performed based on the constructed affine transformation model to obtain corresponding chart data.

[0087] If the number of digital regions cannot reach a preset number threshold, it is considered that the coordinate scale cannot be obtained, and the pixel coordinates are mapped to the data coordinates based on the affine transformation model.

[0088] Affine transformation requires marking two points and their scale values ​​in the image to calculate the affine transformation coefficients. After the calculation is completed, the affine transformation can complete the conversion of pixel coordinates to numerical coordinates. Specifically, in one embodiment, two points P1 and P2 on the X axis and their horizontal coordinate values ​​can be manually marked in the original image through interface interaction. x 1 and x2 , and the two points P3 and P4 on the Y axis and their ordinate values y 1 and y 2 .

[0089] Then the affine transformation of the X axis is:

[0090] ;

[0091] in, represents the affine transformation function of the X-axis, Represents the pixel horizontal coordinate of the point to be mapped, Represents the pixel horizontal coordinate of point P1, Indicates the pixel horizontal coordinate of point P2.

[0092] The affine transformation of the Y axis is:

[0093] ;

[0094] in, represents the affine transformation function of the Y axis, Represents the pixel ordinate of the point to be mapped, Indicates the pixel ordinate of point P3, Indicates the pixel vertical coordinate of point P4.

[0095] In the above specific embodiment, the coordinate scale can be positioned more accurately, the influence of OCR recognition noise is reduced, and the accuracy and robustness of coordinate mapping are improved.

[0096] In one embodiment of the present application, a method for comparing pictures and charts is provided, and chart data obtained based on the picture and chart data acquisition method in any of the above embodiments is compared; the two picture charts compared include: a histogram picture chart and a line chart picture chart, or two histogram picture charts, or two line chart picture charts.

[0097] Based on the obtained chart data, the measurement functions that users can choose include absolute error and relative error, etc. The comparison methods used can include value comparison and trend comparison, etc.

[0098] In one embodiment of the present application, a new trend comparison method is provided. The following is a detailed description of a method for comparing difference points using the new trend comparison method. Please refer to Figure 8 ,include:

[0099] Step S100, obtaining two picture chart data to be compared, forming a data set D with the data coordinates of each key point in one of the picture charts, and forming a data set E with the data coordinates of each key point in the other picture chart.

[0100] The data coordinates in dataset D and dataset E are sorted in ascending order of the X-axis coordinates.

[0101] Step S200: Calculate the predicted value of each data in the data set D based on the first adaptive local fitting model, and calculate the predicted value of each data in the data set E based on the second adaptive local fitting model.

[0102] During the research, the applicant found that in the results of chart data recognition, trend changes in areas with dense key points are often more drastic. If the traditional spline fitting or local linear regression fitting method is used to obtain the predicted value of each data, it is difficult to retain more trend details in areas with dense key points.

[0103] In view of this, this application provides a new adaptive local fitting method, which can adaptively adjust the bandwidth according to the distribution density of key points, so as to retain more trend details in areas with dense key points and maintain the fitting of the overall trend in areas with sparse key points.

[0104] In one embodiment, under the new adaptive local fitting method, the first adaptive local fitting model may include:

[0105] ;

[0106] in, and They represent the intercept and slope of the first adaptive local fitting model, Indicates the first i The horizontal coordinate of the data coordinates, Indicates the first i The predicted value of the ordinate obtained from the abscissa of the data coordinates is the first predicted value.

[0107] in, and The methods for obtaining the loss function include: The gradient descent method is used to iteratively solve the problem so that the loss function Loss is minimized and the optimal solution is obtained. and ;in, Indicates the first j The horizontal coordinate of the data coordinates, J Represents the number of data coordinates in the data set E, 1≤ j ≤ J , Indicates the first j The ordinate of the data coordinates, that is The corresponding truth value; represents the intermediate variable, , exp represents the natural exponential, represents the intermediate variable, , Indicates the default nearest 2 k The span of points, Indicates the first j+k The horizontal coordinate of the data coordinates, Indicates the first j - k The horizontal coordinate of the data coordinates, Indicates a preset minimum positive value.

[0108] Therefore, when solving the first predicted value corresponding to the horizontal coordinate of any data coordinate in the data set D, the horizontal coordinate of any data coordinate is brought into the loss function model based on all data coordinates in the data set E to obtain the required and .

[0109] In one embodiment, the value of k can be any value from 2 to 4.

[0110] Considering the characteristic of sorting in ascending order of X-axis coordinates, The nearest 2 k Therefore, in areas with denser key points, the span of the nearest neighbor points is smaller. Smaller, that is, the bandwidth of local linear regression is smaller, so it can retain more trend details; on the contrary, in areas with sparse key points, the span of the nearest neighbor points is larger, The larger it is, the larger the bandwidth of the local linear regression is, and the model is better able to fit the overall trend.

[0111] The second adaptive local fitting model includes:

[0112] ;

[0113] in, and They represent the intercept term and slope of the second adaptive local fitting model, Indicates the first j The predicted value of the ordinate obtained from the abscissa of the data coordinate is the second predicted value.

[0114] in, and The methods for obtaining the loss function include: The gradient descent method is used to iteratively solve the problem so that the loss function Loss is minimized and the optimal solution is obtained. and ;in, Indicates the first i The ordinate of the data coordinates, that is The corresponding truth value; I Represents the number of data coordinates in the data set D, 1≤ i ≤ I ; represents the intermediate variable, , represents the intermediate variable, , Indicates the default nearest 2 k The span of points, Indicates the first i+k The horizontal coordinate of the data coordinates, Indicates the first i - k The horizontal axis of the data coordinates.

[0115] Therefore, when solving the first predicted value corresponding to the horizontal coordinate of any data coordinate in the data set E, the horizontal coordinate of any data coordinate is brought into the loss function model based on all data coordinates in the data set D to obtain the required and .

[0116] Step S300, calculate the distance between the predicted value and the true value of each data in the data set D as the first distance, and take the data coordinate point whose first distance is greater than the preset distance threshold as the first difference point; calculate the distance between the predicted value and the true value of each data in the data set E as the second distance, and take the data coordinate point whose second distance is greater than the preset distance threshold as the second difference point.

[0117] Step S400, comparing the difference points of two picture charts based on the first difference point and the second difference point.

[0118] Please refer to Fig. 9 ,The left side of the figure is the area with dense key points, where the trend changes dramatically, and the right side is the area with sparse key points, where the trend changes not dramatically. The effect of adaptive fitting is better than spline and local linear regression, achieving a more stable fitting effect.

[0119] Those skilled in the art can understand that Fig. 9 In the figure, the data on the X-axis is the independent variable, and the data on the Y-axis is the dependent variable. This is only an exemplary description of the embodiment of the present application, and can represent a fixed data relationship in any field, and does not have a special meaning.

[0120] In one embodiment, the comparison method further includes: extracting first semantic information and text recognition results based on the parsed chart data, drawing a chart that is consistent with the format of the original image chart, and displaying the parsed data results in the drawn chart.

[0121] In one embodiment of the application, a computer-readable storage medium is provided, on which a program is stored. The stored program includes a method that can be loaded by a processor and process any of the above embodiments.

[0122] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0123] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A method for acquiring picture chart data, characterized in that: include: Collecting chart images; the chart images include images of line graphs and / or images of histograms; Performing chart parsing on the chart image to obtain first semantic information; the first semantic information includes chart type, drawing area and key point pixel coordinates; Performing text recognition on the chart image to obtain a text recognition result; the text recognition result includes the text and the pixel coordinates of the area to which the text belongs; Mapping pixel coordinates to data coordinates in combination with the first semantic information and the text recognition result to obtain chart data; The mapping of pixel coordinates to data coordinates by combining the first semantic information and the text recognition result to obtain chart data includes: Based on the clustering algorithm, the coordinate axis scales are extracted to obtain the densest cluster in the X-axis direction and the densest cluster in the Y-axis direction respectively; It is determined whether the number of digital regions in the most dense cluster in the X-axis direction and the most dense cluster in the Y-axis direction reaches a preset number threshold. If so, the pixel coordinates are mapped to the data coordinates based on the constructed quantile regression model to obtain the corresponding chart data; if not, the pixel coordinates are mapped to the data coordinates based on the constructed affine transformation model to obtain the corresponding chart data; The quantile regression model includes a quantile regression model of the X-axis and a quantile regression model of the Y-axis, and the construction method includes: Collecting chart data with known coordinate scale data, and generating a picture from the chart data; The mean of the horizontal coordinates of the pixel coordinates of the four corner vertices of the quadrilateral area where each coordinate scale pixel position in the image is located is taken as the first mean, the first mean is taken as the independent variable, and the scale value corresponding to the first mean is taken as the dependent variable, and the quantile regression model of the X-axis is trained to obtain the constructed quantile regression model of the X-axis; The mean of the ordinates of the pixel coordinates of the four corner vertices of the quadrilateral area where each coordinate scale pixel position in the image is located is taken as the second mean, the second mean is used as the independent variable, and the scale value corresponding to the second mean is used as the dependent variable to train the quantile regression model of the Y axis to obtain the constructed quantile regression model of the Y axis.

2. The method for acquiring picture chart data according to claim 1, characterized in that: The first semantic information also includes specific type grouping information; the specific type grouping information includes grouping information obtained by grouping key points in a line graph according to their corresponding lines, and / or grouping information obtained by grouping key points in a histogram according to their corresponding rectangles.

3. The method for acquiring picture chart data according to claim 1, characterized in that: The said extraction of coordinate axis scales based on clustering algorithm, respectively obtaining the most dense cluster in the X-axis direction and the most dense cluster in the Y-axis direction, includes: Filter the regions containing numbers in the text recognition results to obtain a region set as a first set; Filter the digital regions in the first set that intersect with the drawing region to form a new digital region set, and use the new digital region set as the second set; Based on the second set, a clustering algorithm is implemented to obtain the most dense cluster in the X-axis direction and the most dense cluster in the Y-axis direction.

4. The method for acquiring picture chart data as claimed in claim 3, characterized in that: The filtering of the regions that are numbers in the text recognition results to obtain a region set as a first set includes: ， Among them, C represents the first set, It represents the mth element in the first set, that is, the mth digital range. M represents the number of digital ranges, 1≤m≤M.

5. The method for acquiring picture chart data according to claim 4, characterized in that: The screening of the digital areas in the first set that intersect with the drawing area includes: for any digital area in the first set, calculating the intersection area between the any digital area and the drawing area, if the intersection area is greater than 0, it means that there is an intersection; the calculating of the intersection area between the any digital area and the drawing area includes: ， in, Represents any numeric range The intersection area with the drawing area; R represents the drawing area, and the pixel coordinates of the drawing area R can be expressed as ,in, Represents the pixel coordinates of the upper left corner of the drawing area R. Represents the pixel coordinates of the upper right corner of the drawing area R, Represents the pixel coordinates of the lower right corner of the drawing area R. Indicates the pixel coordinates of the lower left corner of the drawing area R; any digital area The pixel coordinates can be expressed as ,in, Indicates the digital area The pixel coordinates of the upper left corner of Indicates the digital area The pixel coordinates of the upper right corner of Indicates a digital area The pixel coordinates of the lower right vertex, Indicates the digital area The pixel coordinates of the lower left corner vertex; A is the intermediate variable, .

6. The method for acquiring picture chart data according to claim 3, characterized in that: The clustering algorithm is implemented based on the second set to obtain the most dense cluster in the X-axis direction and the most dense cluster in the Y-axis direction, respectively, including: For the X-axis, the average of the vertical coordinates of the four corner vertices of each digital area in the second set is taken to obtain a third set; based on the third set, a clustering algorithm is implemented to obtain the most dense cluster in the X-axis direction; For the Y-axis, the average of the horizontal coordinates of the four corner vertices of each digital area in the second set is taken to obtain the fourth set; based on the fourth set, a clustering algorithm is implemented to obtain the most dense cluster in the Y-axis direction.

7. A method for comparing pictures and charts, characterized in that: Comparing the chart data obtained based on the method for obtaining picture chart data according to any one of claims 1 to 6; The two image charts to be compared include: a histogram image chart and a line chart image chart, or two histogram image charts, or two line chart image charts.

8. The method for comparing pictures and charts according to claim 7, wherein: The comparison method includes trend comparison, and the method of using trend comparison to compare differences includes: Get two picture charts to be compared, and use the data coordinates of each key point in one of the picture charts to form a data set D, and use the data coordinates of each key point in the other picture chart to form a data set E; wherein the data coordinates in the data sets D and E are sorted in ascending order of the X-axis coordinates; Solve the predicted value of each data in the data set D based on the first adaptive local fitting model, and solve the predicted value of each data in the data set E based on the second adaptive local fitting model; The distance between the predicted value and the true value of each data in the data set D is calculated as the first distance, and the data coordinate point whose first distance is greater than the preset distance threshold is taken as the first difference point; the distance between the predicted value and the true value of each data in the data set E is calculated as the second distance, and the data coordinate point whose second distance is greater than the preset distance threshold is taken as the second difference point; Based on the first difference point and the second difference point, the difference points of the two picture charts are compared.

9. The method for comparing pictures and charts according to claim 8, wherein: The first adaptive local fitting model comprises: ， in, and They represent the intercept and slope of the first adaptive local fitting model, Represents the horizontal coordinate of the i-th data coordinate in the data set D, represents the predicted value of the ordinate based on the abscissa of the i-th data coordinate in the data set D, i.e., the first predicted value; in, and The methods for obtaining the loss function include: The gradient descent method is used to iteratively solve the problem so that the loss function Loss is minimized and the optimal solution is obtained. and ;in, represents the horizontal coordinate of the jth data coordinate in the data set E, J represents the number of data coordinates in the data set E, 1≤j≤J, Represents the ordinate of the jth data coordinate in the data set E, that is, The corresponding truth value; represents the intermediate variable, , exp represents the natural exponential, represents the intermediate variable, , Represents the preset span of the nearest 2k points, Represents the horizontal coordinate of the j+kth data coordinate in the data set E, Represents the horizontal coordinate of the jk-th data coordinate in the data set E, Indicates a preset minimum positive value; The second adaptive local fitting model includes: ， in, and They represent the intercept term and slope of the second adaptive local fitting model, represents the predicted value of the ordinate obtained based on the abscissa of the j-th data coordinate in the data set E, i.e., the second predicted value; in, and The methods for obtaining the loss function include: The gradient descent method is used to iteratively solve the problem so that the loss function Loss is minimized and the optimal solution is obtained. and ;in, Represents the ordinate of the i-th data coordinate in the data set D, that is, The corresponding true value; I represents the number of data coordinates in the data set D, 1≤i≤I; represents the intermediate variable, , represents the intermediate variable, , Represents the preset span of the nearest 2k points, Represents the horizontal coordinate of the i+kth data coordinate in the data set D, Represents the horizontal coordinate of the ikth data coordinate in the data set D.

Images