Information extraction method, device, computer equipment and storage medium for line graph

Abstract

The present application relates to a method, device, computer equipment and storage medium for extracting information from a line graph. The method comprises: by identifying a line graph to be detected, obtaining the attribute information of the coordinate axis of the line graph to be detected and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, and determining the association relationship between the pixel coordinates and the actual coordinates at any position in the line graph to be detected according to the attribute information of the coordinate axis in the line graph to be detected, and determining the actual coordinates corresponding to the data marks of each category in the line graph to be detected based on the association relationship and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark. Thus, a method for automatically extracting information from a line graph based on computer vision technology is realized, which not only improves the efficiency of information extraction, but also ensures the accuracy of information extraction.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, computer equipment and storage medium for extracting information from a line graph. Background Art

[0002] Line charts are basic charts used in many fields. They are often used to observe changes in data over a continuous period of time. They can reflect the trend of things changing over time. In order to express more comprehensive information in line charts, some companies or professional fields usually use multiple data markers to express the categories or meanings of different data.

[0003] In traditional technology, for information extraction of such line graphs with multiple data tags, the most commonly used method is to distinguish the data tags through human observation and estimate the horizontal and vertical axis coordinates of the data tags. However, when extracting information from a batch of such line graphs, extracting information through human observation is not only time-consuming and laborious, but also prone to information extraction errors due to human eye fatigue. Summary of the invention

[0004] Based on this, it is necessary to provide a line graph information extraction method, device, computer equipment and storage medium that can improve the efficiency and accuracy of information extraction in order to address the problem that the extraction of line graph information through human eye observation is time-consuming, labor-intensive and prone to information extraction errors.

[0005] A method for extracting information from a line graph, the method comprising:

[0006] Identify the line graph to be detected, and obtain attribute information of the coordinate axis in the line graph to be detected and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark;

[0007] Determine the correlation between the pixel coordinates at any position in the line graph to be detected and the actual coordinates according to the attribute information of the coordinate axis in the line graph to be detected;

[0008] Based on the association relationship and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, the actual coordinates corresponding to the data marks of each category in the line graph to be detected are determined.

[0009] In one embodiment, the attribute information of the coordinate axis includes the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, as well as the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding vertical coordinate point; the method of determining the association between the pixel coordinates and the actual coordinates at any position in the line graph to be detected based on the attribute information of the coordinate axis in the line graph to be detected includes: determining the association between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate based on the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point; determining the association between the vertical coordinate of the pixel at any position in the line graph to be detected and the actual vertical coordinate based on the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding coordinate point.

[0010] In one of the embodiments, the method of determining the association relationship between the horizontal coordinate of a pixel at any position in the line graph to be detected and the actual horizontal coordinate based on the first pixel coordinate of each coordinate point on the horizontal coordinate in the coordinate axis and the first actual coordinate of the corresponding coordinate point includes: obtaining the first pixel coordinate difference between the first pixel coordinate of any coordinate point on the horizontal coordinate in the coordinate axis and the first pixel coordinate of other coordinate points on the horizontal coordinate, and the first actual coordinate difference between the first actual coordinates corresponding to the two corresponding coordinate points; calculating the first ratio between the first actual coordinate difference and the first pixel coordinate difference, and obtaining the first ratio corresponding to any two coordinate points on the horizontal coordinate in the coordinate axis; for any first ratio, calculating the absolute value of the difference between the first ratio and each other first ratio, and determining the number of times the absolute value is less than a threshold; determining the first ratio with the largest number of times, and the coordinate point with the smallest first pixel coordinate among the coordinate points corresponding to the first ratio; and establishing the association relationship between the horizontal coordinate of a pixel at any position in the line graph to be detected and the actual horizontal coordinate based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate.

[0011] In one embodiment, determining the first ratio with the largest number of times, and the coordinate point with the smallest first pixel coordinate among the coordinate points corresponding to the first ratio, includes: if there are multiple first ratios with the largest number of times, determining the average value of the multiple first ratios as the first ratio with the largest number of times; and based on multiple pairs of coordinate points corresponding to the multiple first ratios, determining the coordinate point with the smallest first pixel coordinate from the multiple pairs of coordinate points.

[0012] In one embodiment, the association relationship between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate is established based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate, including: if the first ratio with the largest number of times is r x , the first pixel coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0', the first actual coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0, the horizontal coordinate of the pixel at any position in the line graph to be detected is x', then for the actual horizontal coordinate x at any position in the line graph to be detected, x=x0+(x'-x0')×r x .

[0013] In one of the embodiments, the method of determining the association relationship between the vertical coordinate of a pixel at any position in the line graph to be detected and the actual vertical coordinate based on the second pixel coordinate of each coordinate point on the vertical coordinate in the coordinate axis and the second actual coordinate of the corresponding coordinate point includes: obtaining the second pixel coordinate difference between the second pixel coordinate of any coordinate point on the vertical coordinate in the coordinate axis and the second pixel coordinate of other coordinate points on the vertical coordinate, and the second actual coordinate difference between the second actual coordinates corresponding to the two corresponding coordinate points; calculating the second ratio between the second actual coordinate difference and the second pixel coordinate difference, and obtaining the second ratio corresponding to any two coordinate points on the vertical coordinate in the coordinate axis; for any second ratio, calculating the absolute value of the difference between the second ratio and each other second ratio, and determining the number of times the absolute value is less than a threshold; determining the second ratio with the largest number of times, and the coordinate point with the smallest second pixel coordinate among the coordinate points corresponding to the second ratio; and establishing the association relationship between the vertical coordinate of a pixel at any position in the line graph to be detected and the actual vertical coordinate based on the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate.

[0014] In one embodiment, determining the second ratio with the largest number of times, and the coordinate point with the smallest second pixel coordinate among the coordinate points corresponding to the second ratio, includes: if there are multiple second ratios with the largest number of times, determining the average value of the multiple second ratios as the second ratio with the largest number of times; based on multiple pairs of coordinate points corresponding to the multiple second ratios, determining the coordinate point with the smallest second pixel coordinate from the multiple pairs of coordinate points.

[0015] In one embodiment, the association relationship between the pixel ordinate at any position in the line graph to be detected and the actual ordinate is established based on the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate, including: if the second ratio with the largest number of times is r y, the second pixel coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0', the second actual coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0, the vertical coordinate of the pixel at any position in the line graph to be detected is y', then for the actual vertical coordinate y at any position in the line graph to be detected, y=y0+(y'-y0')×r y .

[0016] In one of the embodiments, the identifying of the line graph to be detected, obtaining the attribute information of the coordinate axis of the line graph to be detected and the category of each data marker in the line graph to be detected and the pixel coordinates of the corresponding data marker, includes: performing target detection on the line graph to be detected by a pre-trained target detection model to obtain the category of each data marker in the line graph to be detected and the pixel coordinates of the corresponding data marker; performing optical character recognition on the line graph to be detected to obtain the attribute information of the coordinate axis of the line graph to be detected.

[0017] A device for extracting information from a line graph, the device comprising:

[0018] An image detection module is used to identify a line graph to be detected, obtain attribute information of a coordinate axis in the line graph to be detected, and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark;

[0019] An association relationship determination module, used to determine the association relationship between the pixel coordinates at any position in the line graph to be detected and the actual coordinates according to the attribute information of the coordinate axis in the line graph to be detected;

[0020] The actual coordinate determination module is used to determine the actual coordinates corresponding to the data markers of each category in the line graph to be detected based on the association relationship and the category of each data marker in the line graph to be detected and the pixel coordinates of the corresponding data marker.

[0021] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0022] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0023] The above-mentioned line graph information extraction method, device, computer equipment and storage medium, by identifying the line graph to be detected, obtain the attribute information of the coordinate axis of the line graph to be detected and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, and determine the association relationship between the pixel coordinates and the actual coordinates at any position in the line graph to be detected according to the attribute information of the coordinate axis of the line graph to be detected, and determine the actual coordinates corresponding to the data marks of each category in the line graph to be detected based on the association relationship and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark. Thus, a method for automatically extracting information from a line graph based on computer vision technology is realized, which not only improves the efficiency of information extraction, but also ensures the accuracy of information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of a flow chart of a method for extracting information from a line graph in one embodiment;

[0025] Figure 2 A schematic diagram of a flow chart of a step of identifying a line graph to be detected in an embodiment;

[0026] Figure 3 is a schematic diagram of a line graph to be detected in one embodiment;

[0027] Figure 4 In one embodiment, Figure 3 Schematic diagram obtained after target detection;

[0028] Figure 5 A schematic diagram of a line graph to be detected in another embodiment;

[0029] Figure 6 In one embodiment, Figure 5 Schematic diagram obtained after target detection;

[0030] Figure 7 In one embodiment, Figure 3 Schematic diagram obtained after text detection;

[0031] Figure 8 A schematic diagram of performing text recognition in one embodiment;

[0032] Fig. 9 A schematic diagram of a flow chart of a step of determining an association relationship between a pixel horizontal coordinate and an actual horizontal coordinate in one embodiment;

[0033] Fig.10 A schematic diagram of a flow chart of a step of determining an association relationship between a pixel ordinate and an actual ordinate in one embodiment;

[0034] Fig.11is a structural block diagram of a device for extracting information from a line graph in one embodiment;

[0035] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] In one embodiment, Figure 1 As shown, a method for extracting information from a line graph is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0038] Step 102, identifying the line graph to be detected, obtaining the attribute information of the coordinate axis in the line graph to be detected, and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark.

[0039] Among them, the line graph can display continuous data that changes over time (according to the commonly used scale setting). In the line graph, the category data is usually evenly distributed along the horizontal axis, and all value data is evenly distributed along the vertical axis. The line graph to be detected refers to the line graph to be machine-recognized to extract relevant data or information. The attribute information of the coordinate axis includes the position information of each coordinate point on the horizontal axis of the line graph to be detected and the position information of each coordinate point on the vertical axis. Specifically, the position information of each coordinate point includes the actual coordinates of each coordinate point and the pixel coordinates corresponding to the coordinate point. Data tags are tags or identifiers used to distinguish different categories of data or data with different meanings. Pixel coordinates refer to the position of pixels in the image. Since images are composed of pixels, for each data tag in the line graph to be detected, its corresponding pixel coordinates can be found in the image coordinate system. Specifically, the image coordinate system is a direct coordinate system in pixels established with the upper left corner of the image as the origin.

[0040] In this embodiment, machine recognition is performed on the line graph to be detected to obtain the position information of each coordinate point on the horizontal axis and the position information of each coordinate point on the vertical axis in the line graph to be detected, and each data mark in the line graph to be detected, the category of the corresponding data mark and the pixel coordinates of the data mark in the image are obtained.

[0041] Step 104, determining the correlation between the pixel coordinates at any position in the line graph to be detected and the actual coordinates according to the attribute information of the coordinate axis in the line graph to be detected.

[0042] Among them, the pixel coordinates refer to the position of the pixel in the image. The actual coordinates are the coordinates at a certain position determined based on the horizontal coordinates and the vertical coordinates in the line graph to be detected. The association relationship between the pixel coordinates and the actual coordinates can refer to the corresponding relationship or conversion relationship between the pixel coordinates and the actual coordinates. In this embodiment, based on the attribute information of the coordinate axis in the line graph to be detected, the association relationship between the pixel coordinates and the actual coordinates at any position in the line graph to be detected can be determined.

[0043] Step 106, based on the association relationship and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, determine the actual coordinates corresponding to the data marks of each category in the line graph to be detected.

[0044] Specifically, based on the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, as well as the association between the pixel coordinates at any position in the line graph to be detected and the actual coordinates determined above, the actual coordinates corresponding to the data marks of each category in the line graph to be detected can be obtained.

[0045] In the above embodiment, by identifying the line graph to be detected, the attribute information of the coordinate axis of the line graph to be detected and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark are obtained, and according to the attribute information of the coordinate axis of the line graph to be detected, the association relationship between the pixel coordinates at any position in the line graph to be detected and the actual coordinates is determined, and based on the association relationship and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, the actual coordinates corresponding to the data marks of each category in the line graph to be detected are determined. Thus, a method for automatically extracting information from a line graph based on computer vision technology is realized, which not only improves the efficiency of information extraction, but also ensures the accuracy of information extraction.

[0046] In one embodiment, Figure 2 As shown, in the above step 102, the line graph to be detected is identified, and the attribute information of the coordinate axis in the line graph to be detected and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark are obtained, which specifically includes the following steps:

[0047] Step 202, performing target detection on the line graph to be detected by using a pre-trained target detection model, and obtaining the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark.

[0048] The target detection model is a neural network model used to detect the data in the line graph to be detected. It usually needs to undergo customized training of a certain number of samples so that it can have the ability to detect certain types of targets of interest. In this embodiment, by inputting the line graph to be detected into a pre-trained target detection model, the category of the data of interest and the position of the target border in the image can be detected. The border of interest refers to the border corresponding to the detected data.

[0049] for example Figure 3 For the line graph shown in the figure, different types of data need to be represented by different data labels. In the figure, the △ mark represents the data of type A, and the ○ mark represents the data of type B. Assuming that the target detection model that can detect the △ mark and the ○ mark has been trained with this type of samples, Figure 3 The line graph shown is input into the target detection model for prediction, and the category of each data mark and the position of the corresponding data mark in the image can be obtained. For example, if it is a △ mark, it is predicted to be class A, and if it is a ○ mark, it is predicted to be class B. The predicted position information includes the upper left corner pixel coordinates (x1, y1) and the lower right corner pixel coordinates (x2, y2) of the data identification frame. Generally, the origin of the pixel coordinates (x0, y0) is taken at the upper left corner of the image. In this embodiment, the mean of the upper left corner and lower right corner pixel coordinates (i.e. (x1+x2) / 2, (y1+y2) / 2) can be taken as the pixel coordinates of the corresponding data mark.

[0050] Figure 4 It is a visualization effect diagram based on the predicted data marker category and position information (i.e., the pixel coordinates of the corresponding data marker), where the data marker border C is drawn based on the predicted upper left and lower right pixel coordinates of the data marker border, and the letter in the upper left corner of the border C is the predicted data marker category.

[0051] For example Figure 5 For the line graph shown in the figure, the same type of data needs to be labeled with different data to represent different meanings. If the △ mark in the figure represents meaning A, and the ○ mark represents meaning B. Assuming that the target detection model that can detect the △ mark and the ○ mark has been trained with this type of sample, Figure 5 The line graph shown is input into the target detection model for prediction, and the meaning of each data mark and the position of the data mark border in the image can be obtained. If the △ mark is recognized, it is predicted to have meaning A, and if the ○ mark is recognized, it is predicted to have meaning B. The predicted position information includes the upper left pixel coordinates (x1, y1) and the lower right pixel coordinates (x2, y2) of the data mark border. Figure 6As shown, it is a visualization effect diagram based on the predicted data mark meaning and data mark position information, where the border C is drawn based on the upper left corner and lower right corner pixel coordinates of the predicted data mark border, and the letter in the upper left corner of the border C is the meaning of the predicted data mark.

[0052] Step 204, performing optical character recognition on the line graph to be detected, and obtaining attribute information of the coordinate axes of the line graph to be detected.

[0053] The attribute information of the coordinate axis in the line graph to be detected includes the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, and the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding vertical coordinate point. The actual coordinate is the actual coordinate data of each coordinate point on the coordinate axis determined based on the coordinate scale, which is usually marked by a number next to the corresponding coordinate point. Therefore, in this embodiment, optical recognition includes two aspects: text detection and character recognition.

[0054] Specifically, text detection is responsible for detecting the text area in the line graph to be detected and giving the corresponding position information; text recognition is responsible for identifying the detected text area and giving the corresponding text content. Figure 3 Taking the line graph shown in the figure as an example, after performing text detection on the graph, the position of the text area in the image can be obtained, such as Figure 7 As shown, the position information includes the pixel coordinates of the upper left corner (x1, y1), the upper right corner pixel coordinates (x2, y2), the lower left corner pixel coordinates (x3, y3), and the lower right corner pixel coordinates (x4, y4) of the text frame D. In this embodiment, the average value of the pixel coordinates of the four corner points ((x1+x2+x3+x4) / 4, (y1+y2+y3+y4) / 4) is taken as the pixel coordinates of the text area.

[0055] in, Figure 7 It is a visualization effect diagram based on the position information of the text border D predicted after text detection, where the text border D is drawn based on the pixel coordinates of the four corners of the predicted text border. In this embodiment, the text area is cut out from the line graph based on the pixel coordinates of the four corners of the detected text area, and text strip images can be obtained one by one. Then, these text strip images are input into the text recognition algorithm to obtain the text content corresponding to each text strip image. Figure 8 As shown in the figure, the images with "60" and "-10" on the left are two text strip images. After being predicted by the text recognition algorithm, the corresponding text contents "60" and "-10" are obtained, which are the real coordinate values ​​corresponding to the text area, that is, the actual coordinates of the coordinate points on the coordinate axis.

[0056] In one embodiment, the attribute information of the coordinate axis in the line graph to be detected is obtained by the above steps, that is, the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, and the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding vertical coordinate point are obtained. Among them, the first and the second are for distinguishing the coordinate points on different coordinate axes (i.e., the horizontal axis and the vertical axis). Then in the above step 104, according to the attribute information of the coordinate axis in the line graph to be detected, the association relationship between the pixel coordinates and the actual coordinates at any position in the line graph to be detected is determined, specifically including: according to the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, the association relationship between the pixel horizontal coordinate and the actual horizontal coordinate at any position in the line graph to be detected is determined; according to the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding coordinate point, the association relationship between the pixel vertical coordinate and the actual vertical coordinate at any position in the line graph to be detected is determined.

[0057] Since a line graph usually includes a horizontal axis and a vertical axis, and the position of a point on the horizontal axis is uniquely determined by a horizontal axis value, and the position of a point on the vertical axis is uniquely determined by a vertical axis value. A plurality of coordinate points are set at equal intervals with different scales on each coordinate axis according to different actual application scenarios. Therefore, the position of each coordinate point on the coordinate axis determined based on the coordinate scale is the actual coordinate of the corresponding coordinate point. Accordingly, for each coordinate point on the coordinate axis, it has a corresponding pixel coordinate. Based on this, this embodiment determines the correlation between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate according to the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point; determines the correlation between the vertical coordinate of the pixel at any position in the line graph to be detected and the actual vertical coordinate according to the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding coordinate point.

[0058] In one embodiment, Fig. 9 As shown, according to the first pixel coordinate of each coordinate point on the horizontal coordinate of the coordinate axis and the first actual coordinate of the corresponding coordinate point, determining the correlation relationship between the pixel horizontal coordinate and the actual horizontal coordinate at any position in the line graph to be detected includes the following steps:

[0059] Step 902 , obtaining a first pixel coordinate difference between a first pixel coordinate of any coordinate point on the horizontal axis of the coordinate axis and a first pixel coordinate of other coordinate points on the horizontal axis, and a first actual coordinate difference between first actual coordinates corresponding to the two coordinate points.

[0060] Usually, in order to facilitate the marking and reading of data, multiple coordinate points are set at equal intervals on the horizontal axis or the vertical axis, and each coordinate point has a corresponding pixel coordinate and an actual coordinate. In this embodiment, based on the first pixel coordinate of each coordinate point on the horizontal axis in the coordinate axis, the first pixel coordinate difference between the first pixel coordinates of any two coordinate points on the horizontal axis is calculated. Based on the first actual coordinate of each coordinate point on the horizontal axis in the coordinate axis, the first actual coordinate difference between the first actual coordinates of any two coordinate points on the horizontal axis is calculated. That is, the difference between the first pixel coordinate of any coordinate point on the horizontal axis and the first pixel coordinate of each other coordinate point on the horizontal axis is calculated, so as to obtain the first pixel coordinate difference between any two coordinate points on the horizontal axis. The difference between the first actual coordinate of any coordinate point on the horizontal axis and the first actual coordinate of each other coordinate point on the horizontal axis is calculated, so as to obtain the first actual coordinate difference between any two coordinate points on the horizontal axis.

[0061] Step 904 , calculating a first ratio between the first actual coordinate difference and the first pixel coordinate difference, and obtaining a first ratio corresponding to any two coordinate points on the horizontal coordinate of the coordinate axis.

[0062] Specifically, for any two coordinate points on the horizontal axis, the ratio between the first actual coordinate difference and the first pixel coordinate difference between the two coordinate points is calculated to obtain the first ratio corresponding to the two coordinate points. For all coordinate points on the horizontal axis in the line graph to be detected, the first ratio corresponding to any two coordinate points can be calculated respectively.

[0063] Step 906: For any first ratio, calculate the absolute value of the difference between the first ratio and each of the other first ratios, and determine the number of times the absolute value is less than a threshold.

[0064] The threshold is a preset number that tends to zero. In this embodiment, for the first ratio of any two coordinate points on the horizontal axis of the line graph to be detected obtained above, the difference between any first ratio and each other first ratio is calculated, and the absolute value of the difference is determined, and the absolute value is compared with the threshold to determine the number of times the absolute value is less than the threshold.

[0065] For example, if there are four coordinate points X1, X2, X3 and X4 on the horizontal axis of the line graph to be detected, if the first ratio of coordinate points X1 and X2 is Rx12, where Rx12 is the ratio between the actual coordinate difference and the pixel coordinate difference between coordinate points X1 and X2. Then the first ratio of coordinate points X1 and X3 is Rx13, the first ratio of coordinate points X1 and X4 is Rx14, the first ratio of coordinate points X2 and X3 is Rx23, the first ratio of coordinate points X2 and X4 is Rx24, and the first ratio of coordinate points X3 and X4 is Rx34. For any first ratio Rx12, calculate the absolute value of the difference between it and each other first ratio, that is, calculate |Rx12-Rx13|, |Rx12-Rx14|, |Rx12-Rx23|, |Rx12-Rx24| and |Rx12-Rx34| respectively, and compare the absolute value obtained by each calculation with the threshold, and count the number of times the absolute value is less than the threshold. For example, for the first ratio Rx12, if 3 of the 5 absolute values ​​calculated above are less than the threshold, then for the first ratio Rx12, the number of times its corresponding absolute value is less than the threshold is 3. Similarly, for the first ratios corresponding to any two coordinate points on the horizontal axis of the line graph to be detected, the number of times its corresponding absolute value is less than the threshold can be determined.

[0066] Step 908, determining the first ratio with the largest number of occurrences and the coordinate point with the smallest first pixel coordinate among the coordinate points corresponding to the first ratio.

[0067] Based on the above calculated data, the first ratio whose corresponding absolute value is less than the threshold value the most times can be determined. Since each first ratio involves two coordinate points, in this embodiment, the coordinate point with the smallest first pixel coordinate is determined from the first pixel coordinates corresponding to the two coordinate points involved in the first ratio with the most times.

[0068] Furthermore, if there are multiple first ratios with the largest number of times, the average value of the multiple first ratios is determined as the first ratio with the largest number of times. Then, based on the multiple pairs of coordinate points corresponding to the multiple first ratios, the coordinate point with the smallest first pixel coordinate is determined from the multiple pairs of coordinate points.

[0069] Step 910, based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate, establish an association relationship between the pixel horizontal coordinate at any position in the line graph to be detected and the actual horizontal coordinate.

[0070] For example, if the first ratio with the largest number of times is r x, the first pixel coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0', the first actual coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0, the horizontal coordinate of the pixel at any position in the line graph to be detected is x', then for the actual horizontal coordinate x at any position in the line graph to be detected, x=x0+(x'-x0')×r x Since r is known from the above steps x , x0' and x0, then for the pixel horizontal coordinate x' at any position in the line graph to be detected, the actual horizontal coordinate x at that position can be quickly calculated through the above relationship, that is, the actual horizontal coordinate value at that position can be obtained.

[0071] In one embodiment, Fig.10 As shown, according to the second pixel coordinate of each coordinate point on the ordinate in the coordinate axis and the second actual coordinate of the corresponding coordinate point, determining the correlation relationship between the pixel ordinate at any position in the line graph to be detected and the actual ordinate includes the following steps:

[0072] Step 1002, obtaining the second pixel coordinate difference between the second pixel coordinate of any coordinate point on the ordinate in the coordinate axis and the second pixel coordinate of other coordinate points on the ordinate, and the second actual coordinate difference between the second actual coordinates corresponding to the two coordinate points.

[0073] Step 1004, calculating a second ratio between the second actual coordinate difference and the second pixel coordinate difference, and obtaining the second ratio corresponding to any two coordinate points on the ordinate in the coordinate axis.

[0074] Step 1006: For any second ratio, calculate the absolute value of the difference between the second ratio and each other second ratio, and determine the number of times the absolute value is less than a threshold.

[0075] Step 1008, determining the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate among the coordinate points corresponding to the second ratio.

[0076] Furthermore, if there are multiple second ratios with the largest number of times, the average value of the multiple second ratios is determined as the second ratio with the largest number of times, and based on the multiple pairs of coordinate points corresponding to the multiple second ratios, the coordinate point with the smallest second pixel coordinate is determined from the multiple pairs of coordinate points.

[0077] Step 1010, based on the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate, establish an association relationship between the pixel vertical coordinate at any position in the line graph to be detected and the actual vertical coordinate.

[0078] For example, if the second ratio with the highest number of times is r y, the second pixel coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0', the second actual coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0, and the vertical coordinate of the pixel at any position in the line graph to be detected is y', then for the actual vertical coordinate y at any position in the line graph to be detected, y=y0+(y'-y0')×r y Since r is known from the above steps y , y0' and y0, then for the pixel horizontal coordinate y' at any position in the line graph to be detected, the actual vertical coordinate y at that position can be quickly calculated through the above relationship, that is, the actual vertical coordinate value at that position can be obtained.

[0079] It is understandable that the specific implementation process of the above embodiment is different from Fig. 9 The embodiments shown are similar and will not be described again in detail.

[0080] It should be understood that although Figure 1-Figure 10 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-Figure 10 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0081] In one embodiment, Fig.11 As shown, a device for extracting information from a line graph is provided, comprising: an image detection module 1101, an association relationship determination module 1102 and an actual coordinate determination module 1103, wherein:

[0082] The image detection module 1101 is used to identify the line graph to be detected, obtain the attribute information of the coordinate axis in the line graph to be detected, and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark;

[0083] An association relationship determination module 1102 is used to determine the association relationship between the pixel coordinates at any position in the line graph to be detected and the actual coordinates according to the attribute information of the coordinate axis in the line graph to be detected;

[0084] The actual coordinate determination module 1103 is used to determine the actual coordinates corresponding to the data markers of each category in the line graph to be detected based on the association relationship and the category of each data marker in the line graph to be detected and the pixel coordinates of the corresponding data marker.

[0085] In one embodiment, the attribute information of the coordinate axis includes the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, as well as the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding vertical coordinate point; the association relationship determination module specifically includes: a horizontal axis association relationship determination unit, which is used to determine the association relationship between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate according to the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point; a vertical coordinate association relationship determination unit, which is used to determine the association relationship between the vertical coordinate of the pixel at any position in the line graph to be detected and the actual vertical coordinate according to the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding coordinate point.

[0086] In one embodiment, the horizontal coordinate association relationship determination unit is specifically used to: obtain the first pixel coordinate difference between the first pixel coordinate of any coordinate point on the horizontal coordinate of the coordinate axis and the first pixel coordinate of other coordinate points on the horizontal coordinate, and the first actual coordinate difference between the first actual coordinates corresponding to the two coordinate points; calculate the first ratio between the first actual coordinate difference and the first pixel coordinate difference, and obtain the first ratio corresponding to any two coordinate points on the horizontal coordinate of the coordinate axis; for any first ratio, calculate the absolute value of the difference between the first ratio and each other first ratio, and determine the number of times the absolute value is less than a threshold; determine the first ratio with the largest number of times, and the coordinate point with the smallest first pixel coordinate among the coordinate points corresponding to the first ratio; based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate, establish an association relationship between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate.

[0087] In one embodiment, the horizontal coordinate association relationship determination unit is also used for: if there are multiple first ratios with the largest number of occurrences, then the average value of the multiple first ratios is determined as the first ratio with the largest number of occurrences; based on multiple pairs of coordinate points corresponding to the multiple first ratios, respectively, determining the coordinate point with the smallest first pixel coordinate from the multiple pairs of coordinate points.

[0088] In one embodiment, the horizontal coordinate association relationship determination unit is further configured to: if the first ratio with the largest number of times is r x, the first pixel coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0', the first actual coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0, the horizontal coordinate of the pixel at any position in the line graph to be detected is x', then for the actual horizontal coordinate x at any position in the line graph to be detected, x=x0+(x'-x0')×r x .

[0089] In one embodiment, the vertical coordinate association relationship determination unit is specifically used to: obtain the second pixel coordinate difference between the second pixel coordinate of any coordinate point on the vertical coordinate in the coordinate axis and the second pixel coordinate of other coordinate points on the vertical coordinate, and the second actual coordinate difference between the second actual coordinates corresponding to the two coordinate points; calculate the second ratio between the second actual coordinate difference and the second pixel coordinate difference, and obtain the second ratio corresponding to any two coordinate points on the vertical coordinate in the coordinate axis; for any second ratio, calculate the absolute value of the difference between the second ratio and each other second ratio, and determine the number of times the absolute value is less than a threshold; determine the second ratio with the largest number of times, and the coordinate point with the smallest second pixel coordinate among the coordinate points corresponding to the second ratio; based on the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate, establish an association relationship between the pixel vertical coordinate and the actual vertical coordinate at any position in the line graph to be detected.

[0090] In one embodiment, the vertical coordinate association relationship determination unit is also used for: if there are multiple second ratios with the largest number of times, then the average value of the multiple second ratios is determined as the second ratio with the largest number of times; based on multiple pairs of coordinate points corresponding to the multiple second ratios, respectively, determining the coordinate point with the smallest second pixel coordinate from the multiple pairs of coordinate points.

[0091] In one embodiment, the vertical coordinate association relationship determination unit is further configured to: if the second ratio with the largest number of times is r y , the second pixel coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0', the second actual coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0, the vertical coordinate of the pixel at any position in the line graph to be detected is y', then for the actual vertical coordinate y at any position in the line graph to be detected, y=y0+(y'-y0')×r y .

[0092] In one embodiment, the image detection module is specifically used to: perform target detection on the line graph to be detected through a pre-trained target detection model to obtain the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark; perform optical character recognition on the line graph to be detected to obtain the attribute information of the coordinate axis in the line graph to be detected.

[0093] The specific definition of the information extraction device for the line graph can be found in the definition of the information extraction method for the line graph above, which will not be repeated here. The various modules in the above-mentioned information extraction device for the line graph can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0094] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for extracting information from a line graph is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the computer device housing, or an external keyboard, touch pad or mouse, etc.

[0095] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0096] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0097] Identify the line graph to be detected, and obtain attribute information of the coordinate axis in the line graph to be detected and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark;

[0098] Determine the correlation between the pixel coordinates at any position in the line graph to be detected and the actual coordinates according to the attribute information of the coordinate axis in the line graph to be detected;

[0099] Based on the association relationship and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, the actual coordinates corresponding to the data marks of each category in the line graph to be detected are determined.

[0100] In one embodiment, the attribute information of the coordinate axis includes the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, as well as the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding vertical coordinate point; when the processor executes the computer program, the following steps are also implemented: according to the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, the association relationship between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate is determined; according to the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding coordinate point, the association relationship between the vertical coordinate of the pixel at any position in the line graph to be detected and the actual vertical coordinate is determined.

[0101] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the first pixel coordinate difference between the first pixel coordinate of any coordinate point on the horizontal coordinate of the coordinate axis and the first pixel coordinate of other coordinate points on the horizontal coordinate, and the first actual coordinate difference between the first actual coordinates corresponding to the two coordinate points; calculating the first ratio between the first actual coordinate difference and the first pixel coordinate difference, and obtaining the first ratio corresponding to any two coordinate points on the horizontal coordinate of the coordinate axis; for any first ratio, calculating the absolute value of the difference between the first ratio and each other first ratio, and determining the number of times the absolute value is less than a threshold; determining the first ratio with the largest number of times, and the coordinate point with the smallest first pixel coordinate among the coordinate points corresponding to the first ratio; based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate, establishing an association relationship between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate.

[0102] In one embodiment, when the processor executes the computer program, the following steps are also implemented: if there are multiple first ratios with the largest number of times, the average value of the multiple first ratios is determined as the first ratio with the largest number of times; based on multiple pairs of coordinate points corresponding to the multiple first ratios, the coordinate point with the smallest first pixel coordinate is determined from the multiple pairs of coordinate points.

[0103] In one embodiment, when the processor executes the computer program, the following steps are further implemented: if the first ratio with the largest number of times is r x, the first pixel coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0', the first actual coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0, the horizontal coordinate of the pixel at any position in the line graph to be detected is x', then for the actual horizontal coordinate x at any position in the line graph to be detected, x=x0+(x'-x0')×r x .

[0104] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the second pixel coordinate difference between the second pixel coordinate of any coordinate point on the vertical coordinate in the coordinate axis and the second pixel coordinate of other coordinate points on the vertical coordinate, and the second actual coordinate difference between the second actual coordinates corresponding to the two coordinate points; calculating the second ratio between the second actual coordinate difference and the second pixel coordinate difference, and obtaining the second ratio corresponding to any two coordinate points on the vertical coordinate in the coordinate axis; for any second ratio, calculating the absolute value of the difference between the second ratio and each other second ratio, and determining the number of times the absolute value is less than a threshold; determining the second ratio with the largest number of times, and the coordinate point with the smallest second pixel coordinate among the coordinate points corresponding to the second ratio; based on the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate, establishing an association relationship between the pixel vertical coordinate and the actual vertical coordinate at any position in the line graph to be detected.

[0105] In one embodiment, when the processor executes the computer program, the following steps are also implemented: if there are multiple second ratios with the largest number of times, the average value of the multiple second ratios is determined as the second ratio with the largest number of times; based on multiple pairs of coordinate points corresponding to the multiple second ratios, the coordinate point with the smallest second pixel coordinate is determined from the multiple pairs of coordinate points.

[0106] In one embodiment, when the processor executes the computer program, the following steps are further implemented: if the second ratio with the largest number of times is r y , the second pixel coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0', the second actual coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0, the vertical coordinate of the pixel at any position in the line graph to be detected is y', then for the actual vertical coordinate y at any position in the line graph to be detected, y=y0+(y'-y0')×r y .

[0107] In one embodiment, when the processor executes the computer program, the following steps are also implemented: performing target detection on the line graph to be detected through a pre-trained target detection model to obtain the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark; performing optical character recognition on the line graph to be detected to obtain the attribute information of the coordinate axis in the line graph to be detected.

[0108] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0109] Identify the line graph to be detected, and obtain attribute information of the coordinate axis in the line graph to be detected and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark;

[0110] Determine the correlation between the pixel coordinates at any position in the line graph to be detected and the actual coordinates according to the attribute information of the coordinate axis in the line graph to be detected;

[0111] Based on the association relationship and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, the actual coordinates corresponding to the data marks of each category in the line graph to be detected are determined.

[0112] In one embodiment, the attribute information of the coordinate axis includes the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, as well as the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding vertical coordinate point; when the computer program is executed by the processor, the following steps are also implemented: according to the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, the association relationship between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate is determined; according to the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding coordinate point, the association relationship between the vertical coordinate of the pixel at any position in the line graph to be detected and the actual vertical coordinate is determined.

[0113] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the first pixel coordinate difference between the first pixel coordinate of any coordinate point on the horizontal coordinate of the coordinate axis and the first pixel coordinate of other coordinate points on the horizontal coordinate, and the first actual coordinate difference between the first actual coordinates corresponding to the two coordinate points; calculating the first ratio between the first actual coordinate difference and the first pixel coordinate difference, and obtaining the first ratio corresponding to any two coordinate points on the horizontal coordinate of the coordinate axis; for any first ratio, calculating the absolute value of the difference between the first ratio and each other first ratio, and determining the number of times the absolute value is less than a threshold; determining the first ratio with the largest number of times, and the coordinate point with the smallest first pixel coordinate among the coordinate points corresponding to the first ratio; based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate, establishing an association relationship between the horizontal coordinate of the pixel at any position in the line graph to be detected and the actual horizontal coordinate.

[0114] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: if there are multiple first ratios with the largest number of times, the average value of the multiple first ratios is determined as the first ratio with the largest number of times; based on multiple pairs of coordinate points corresponding to the multiple first ratios, the coordinate point with the smallest first pixel coordinate is determined from the multiple pairs of coordinate points.

[0115] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: if the first ratio with the largest number of times is r x , the first pixel coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0', the first actual coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x0, the horizontal coordinate of the pixel at any position in the line graph to be detected is x', then for the actual horizontal coordinate x at any position in the line graph to be detected, x=x0+(x'-x0')×r x .

[0116] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the second pixel coordinate difference between the second pixel coordinate of any coordinate point on the vertical coordinate in the coordinate axis and the second pixel coordinate of other coordinate points on the vertical coordinate, and the second actual coordinate difference between the second actual coordinates corresponding to the two coordinate points; calculating the second ratio between the second actual coordinate difference and the second pixel coordinate difference, and obtaining the second ratio corresponding to any two coordinate points on the vertical coordinate in the coordinate axis; for any second ratio, calculating the absolute value of the difference between the second ratio and each other second ratio, and determining the number of times the absolute value is less than a threshold; determining the second ratio with the largest number of times, and the coordinate point with the smallest second pixel coordinate among the coordinate points corresponding to the second ratio; based on the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate, establishing an association relationship between the pixel vertical coordinate and the actual vertical coordinate at any position in the line graph to be detected.

[0117] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: if there are multiple second ratios with the largest number of times, the average value of the multiple second ratios is determined as the second ratio with the largest number of times; based on multiple pairs of coordinate points corresponding to the multiple second ratios, the coordinate point with the smallest second pixel coordinate is determined from the multiple pairs of coordinate points.

[0118] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: if the second ratio with the largest number of times is r y, the second pixel coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0', the second actual coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y0, the vertical coordinate of the pixel at any position in the line graph to be detected is y', then for the actual vertical coordinate y at any position in the line graph to be detected, y=y0+(y'-y0')×r y .

[0119] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing target detection on the line graph to be detected through a pre-trained target detection model to obtain the category of each data marker in the line graph to be detected and the pixel coordinates of the corresponding data marker; performing optical character recognition on the line graph to be detected to obtain the attribute information of the coordinate axis in the line graph to be detected.

[0120] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0121] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for extracting information from a line graph. It is characterized in that Different categories of data in the line graph are represented by different data tags, where the data tags are tags or identifiers used to distinguish different categories of data or data with different meanings. The method includes: Performing target detection on the line graph to be detected by using a pre-trained target detection model, obtaining the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, wherein the pixel coordinates of the data mark are the average of the upper left corner pixel coordinates and the lower right corner pixel coordinates of the border of the data mark predicted by the target detection model; Performing optical character recognition on the line graph to be detected to obtain attribute information of a coordinate axis in the line graph to be detected, wherein the attribute information of the coordinate axis includes a first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and a first actual coordinate of the corresponding coordinate point, and a second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and a second actual coordinate of the corresponding vertical coordinate point; Determine the correlation between the pixel coordinates at any position in the line graph to be detected and the actual coordinates according to the attribute information of the coordinate axis in the line graph to be detected; Based on the association relationship and the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, determine the actual coordinates corresponding to the data marks of each category in the line graph to be detected; The determining, based on the attribute information of the coordinate axis in the line graph to be detected, the association relationship between the pixel coordinates at any position in the line graph to be detected and the actual coordinates comprises: Obtain the first pixel coordinate difference between the first pixel coordinate of any coordinate point on the horizontal coordinate of the coordinate axis and the first pixel coordinate of other coordinate points on the horizontal coordinate, and the first actual coordinate difference between the first actual coordinates corresponding to the two coordinate points; calculate the first ratio between the first actual coordinate difference and the first pixel coordinate difference, and obtain the first ratio corresponding to any two coordinate points on the horizontal coordinate of the coordinate axis; for any first ratio, calculate the absolute value of the difference between the first ratio and each other first ratio, and determine the number of times the absolute value is less than a threshold; determine the first ratio with the largest number of times, and the coordinate point with the smallest first pixel coordinate among the coordinate points corresponding to the first ratio; based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate, establish an association relationship between the pixel horizontal coordinate and the actual horizontal coordinate at any position in the line graph to be detected; According to the second pixel coordinate of each coordinate point on the ordinate in the coordinate axis and the second actual coordinate of the corresponding coordinate point, the correlation relationship between the pixel ordinate and the actual ordinate at any position in the line graph to be detected is determined.

2. The method according to claim 1, It is characterized in that The determining of the first ratio having the largest number of times and the coordinate point having the smallest first pixel coordinate among the coordinate points corresponding to the first ratio includes: If there are multiple first ratios with the largest number of times, an average value of the multiple first ratios is determined as the first ratio with the largest number of times; Based on the multiple pairs of coordinate points corresponding to the multiple first ratios respectively, a coordinate point with the smallest first pixel coordinate is determined from the multiple pairs of coordinate points.

3. The method according to claim 1, It is characterized in that The establishing of an association relationship between the horizontal coordinate of a pixel at any position in the line graph to be detected and the actual horizontal coordinate based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate comprises: If the first ratio with the largest number of times is r x , the first pixel coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x 0 ', the first actual coordinate corresponding to the coordinate point with the smallest first pixel coordinate is x 0 , the pixel horizontal coordinate at any position in the line graph to be detected is x', then the actual horizontal coordinate x at any position in the line graph to be detected is x=x 0 +(x'-x 0 ')×r x .

4. The method according to claim 1, It is characterized in that Determining the correlation between the pixel ordinate and the actual ordinate at any position in the line graph to be detected according to the second pixel coordinate of each coordinate point on the ordinate in the coordinate axis and the second actual coordinate of the corresponding coordinate point comprises: Obtain a second pixel coordinate difference between a second pixel coordinate of any coordinate point on the ordinate in the coordinate axis and a second pixel coordinate difference between second pixel coordinates of other coordinate points on the ordinate, and a second actual coordinate difference between second actual coordinates corresponding to the two coordinate points; Calculate a second ratio between the second actual coordinate difference and the second pixel coordinate difference, and obtain the second ratio corresponding to any two coordinate points on the ordinate in the coordinate axis; For any second ratio, calculate the absolute value of the difference between the second ratio and each other second ratio, and determine the number of times the absolute value is less than a threshold; Determine the second ratio with the largest number of times, and the coordinate point with the smallest second pixel coordinate among the coordinate points corresponding to the second ratio; Based on the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate, an association relationship between the pixel vertical coordinate at any position in the line graph to be detected and the actual vertical coordinate is established.

5. The method according to claim 4, It is characterized in that The determining of the second ratio having the largest number of times and the coordinate point having the smallest second pixel coordinate among the coordinate points corresponding to the second ratio includes: If there are multiple second ratios with the largest number of times, an average value of the multiple second ratios is determined as the second ratio with the largest number of times; Based on the multiple pairs of coordinate points corresponding to the multiple second ratios respectively, a coordinate point with the smallest second pixel coordinate is determined from the multiple pairs of coordinate points.

6. The method according to claim 4, It is characterized in that The establishing of an association relationship between the pixel ordinate at any position in the line graph to be detected and the actual ordinate based on the second ratio with the largest number of times and the coordinate point with the smallest second pixel coordinate comprises: If the second ratio with the largest number of times is r y , the second pixel coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y 0 ', the second actual coordinate corresponding to the coordinate point with the smallest second pixel coordinate is y 0 , the vertical coordinate of the pixel at any position in the line graph to be detected is y', then the actual vertical coordinate y at any position in the line graph to be detected is y=y 0 +(y'-y 0 ')×r y .

7. A device for extracting information from a line graph, It is characterized in that Different categories of data in the line graph are represented by different data marks, and the data marks are marks or identifiers used to distinguish different categories of data or data with different meanings. The device includes: An image detection module is used to perform target detection on a line graph to be detected through a pre-trained target detection model, obtain the category of each data mark in the line graph to be detected and the pixel coordinates of the corresponding data mark, wherein the pixel coordinates of the data mark are the average of the upper left corner pixel coordinates and the lower right corner pixel coordinates of the border of the data mark predicted by the target detection model; and perform optical character recognition on the line graph to be detected to obtain attribute information of the coordinate axis in the line graph to be detected, wherein the attribute information of the coordinate axis includes the first pixel coordinate of each coordinate point on the horizontal axis of the coordinate axis and the first actual coordinate of the corresponding coordinate point, and the second pixel coordinate of each coordinate point on the vertical axis of the coordinate axis and the second actual coordinate of the corresponding vertical coordinate point; An association relationship determination module, used to determine the association relationship between the pixel coordinates at any position in the line graph to be detected and the actual coordinates according to the attribute information of the coordinate axis in the line graph to be detected; An actual coordinate determination module, used to determine the actual coordinates corresponding to the data markers of each category in the line graph to be detected based on the association relationship and the category of each data marker in the line graph to be detected and the pixel coordinates of the corresponding data marker; The association relationship determination module is specifically used for: Obtain the first pixel coordinate difference between the first pixel coordinate of any coordinate point on the horizontal coordinate of the coordinate axis and the first pixel coordinate of other coordinate points on the horizontal coordinate, and the first actual coordinate difference between the first actual coordinates corresponding to the two coordinate points; calculate the first ratio between the first actual coordinate difference and the first pixel coordinate difference, and obtain the first ratio corresponding to any two coordinate points on the horizontal coordinate of the coordinate axis; for any first ratio, calculate the absolute value of the difference between the first ratio and each other first ratio, and determine the number of times the absolute value is less than a threshold; determine the first ratio with the largest number of times, and the coordinate point with the smallest first pixel coordinate among the coordinate points corresponding to the first ratio; based on the first ratio with the largest number of times and the coordinate point with the smallest first pixel coordinate, establish an association relationship between the pixel horizontal coordinate and the actual horizontal coordinate at any position in the line graph to be detected; According to the second pixel coordinate of each coordinate point on the ordinate in the coordinate axis and the second actual coordinate of the corresponding coordinate point, the correlation relationship between the pixel ordinate and the actual ordinate at any position in the line graph to be detected is determined.

8. The device according to claim 7, It is characterized in that The association relationship determination module is also used for: If there are multiple first ratios with the largest number of times, an average value of the multiple first ratios is determined as the first ratio with the largest number of times; Based on the multiple pairs of coordinate points corresponding to the multiple first ratios respectively, a coordinate point with the smallest first pixel coordinate is determined from the multiple pairs of coordinate points.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images