A polyline chart data reverse extraction method and system based on differentiable rendering

Abstract

This application discloses a method and system for reverse extraction of line graph data based on differentiable rendering, comprising the following six steps: (1) the user uploads the line graph image from which data needs to be extracted; (2) the system preprocesses the line graph image; (3) the system constructs the initial vector path of the target line graph; (4) the system iteratively optimizes the vector path based on the differentiable rendering mechanism; (5) the system parses the coordinate axes and scale information to establish the pixel-to-value mapping relationship; (6) the system samples the optimized vector path and outputs structured numerical data. The method proposed in this invention can reconstruct continuous data curves through parametric vector modeling, and achieve sub-pixel alignment of geometric paths by using differentiable rendering and adaptive piecewise weighted loss function. It effectively overcomes the jagged effect of traditional methods and preserves high-frequency fluctuation details, achieving high-fidelity data reverse restoration in complex backgrounds and improving the accuracy of recovering continuous numerical data from raster images.

Description

Technical Field

[0001] This application relates to the fields of computer vision, data visualization understanding and processing technology, and in particular to a method and system for reverse extraction of line graph data based on differentiable rendering. Background Technology

[0002] Line charts are an important form of data visualization, widely used in scientific research, financial analysis, and statistical reporting. Visualizing raw numerical data as line charts helps users intuitively understand data trends and improves data analysis efficiency. However, in practical applications, many line charts exist in image form, and the underlying numerical data is difficult to obtain directly after the image is generated, thus limiting further data reuse and analysis.

[0003] Various image data extraction schemes have been proposed in existing technologies. These technologies can be broadly categorized into three types: the first type relies on manual or semi-automatic methods, using user interaction to annotate or correct line graphs point by point to achieve data recovery; the second type uses rule-based image processing methods that rely on color, edge, or geometric features for line graph detection and reconstruction; and the third type uses deep learning methods to identify line graph pixels or discrete key points and recover the corresponding data. These three types of technologies have, to some extent, achieved the goal of extracting numerical information from line graph images, reducing the cost of manually reading chart data. However, existing research on line graph data extraction typically treats lines as objects composed of discrete pixels or independent points, focusing primarily on local pixel-level or point-level matching, without fully considering the continuous function characteristics of lines. In practical applications, this discrete modeling approach is easily affected by image resolution, noise, and curve smoothness, making it difficult to accurately depict the true shape of smooth curves and high-frequency changing lines. On the other hand, image vectorization techniques attempt to convert bitmap images into continuous vector representations to obtain better geometric expression capabilities. However, existing vectorization methods are mostly designed for general graphics processing and lack modeling for the semantics of line chart data. They are prone to path redundancy, breaks or incorrect connections, and are difficult to use directly for reverse extraction of high-precision numerical data.

[0004] Therefore, existing line chart data extraction techniques still have shortcomings in continuous geometric modeling and numerical accuracy, and are still unable to meet the needs of high-precision, automated reverse extraction of complex line chart data. Summary of the Invention

[0005] To address the shortcomings of existing techniques, this paper proposes a method and system for reverse extraction of line graph data based on differentiable rendering. This method establishes a continuous vectorized representation model for the data curves in the line graph, thereby accurately depicting the continuous geometric characteristics of the line. Simultaneously, the method models the line graph data extraction process as an optimizable rendering process, iteratively optimizing the vector curves in pixel space to maintain a high degree of consistency with the original line graph image, thus improving the accuracy of numerical data recovery. Furthermore, since line graph reconstruction is susceptible to noise and resolution limitations, this method introduces a vector path initialization and optimization mechanism to automatically extract line graph data without manual intervention. It also combines coordinate axis and scale information to complete numerical mapping, improving the efficiency and reliability of reverse extraction of line graph data.

[0006] To solve the technical problem, the technical solution of the present invention is as follows:

[0007] A method for reverse extraction of line chart data based on differentiable rendering, the method comprising:

[0008] S1: Obtain the line chart image to be processed uploaded by the user;

[0009] S2: Preprocess the line graph image to eliminate interference from non-data elements and extract the pixel image of a single target data line as the target image;

[0010] S3: Convert the discrete raster pixels in the pixel map extracted in step S2 into an initial vector path composed of a sequence of parameterized cubic Bézier curves.

[0011] S4: Based on the differentiable rendering mechanism, the control point coordinates and stroke width of the initialized vector path described in step S3 are used as optimizable parameters. A prediction image is generated through a differentiable renderer. The mixed loss function between the prediction image and the target image described in step S2 is calculated, and the gradient is backpropagated to iteratively update the optimizable parameters to obtain the optimized vector path. The mixed loss function is composed of a global loss and an adaptive segmented loss. The adaptive segmented loss divides the path into multiple segments and dynamically increases the weight of high-error segments.

[0012] S5: Analyze the coordinate axes and scale information of the line graph image to establish a mapping relationship from pixel coordinates to actual values;

[0013] S6: Sample the vector path optimized in step S4, use the mapping relationship established in step S5 to convert the pixel coordinates of the sampled points into numerical data, and output them as structured data format.

[0014] Furthermore, the preprocessing in step S2 includes:

[0015] S21: Text region detection and removal. First, text regions in the image are identified through multi-feature fusion and geometric screening. Then, the background is filled with a Telea-based image inpainting algorithm to remove the text, resulting in a text-free image.

[0016] S22: Data region boundary positioning: On the text-free image, directional morphological filtering and projection contour analysis are used to locate the actual data line region and crop out the effective data line region image.

[0017] S23: Separation and extraction of a single data line: The effective data line region image is converted from RGB color space to CIELAB color space. K-means clustering is used to identify the main foreground color and generate a color prototype. A binary mask is generated based on color similarity. After anisotropic morphological operations and connected component analysis, the pixel image of the single target data line is extracted as the target image.

[0018] Furthermore, step S3, converting discrete raster pixels into initial vector paths, includes: applying Gaussian smoothing to the pixel image, using the Canny operator to detect edges and extract the maximum connected contour to obtain an ordered set of discrete pixels. , For the first The coordinates of a discrete pixel;

[0019] A recursive subdivision algorithm is used to estimate the parameters of each point by parameterizing the chord length and then solve for the control points using the linear least squares method. The ordered discrete pixel set is fitted into cubic Bézier curve segments; the fitting error of each segment is calculated. ,when hour, To set a preset threshold, the point set is split into two parts at the midpoint of the segment, and sub-segments are recursively fitted until the error of all segments meets the requirements; the cubic Bézier curve segments are then connected end to end to obtain the initial vector path. , For the first Each Bézier curve segment is initialized with its control point coordinates and uniform stroke width.

[0020] Furthermore, in step S4, the differentiable renderer, during each iteration of forward propagation, adjusts the parameters based on the current optimizable parameters. and stroke width Generate predictive raster images The hybrid loss function is:

[0021]

[0022] in, To predict raster images With the target image The mean square error of the whole image. For adaptive piecewise loss, The ratio of the adaptive piecewise loss function; the adaptive piecewise loss divides the path into... For each Bézier segment, a region of interest is defined based on its control points, and the local error is calculated. The weight of high-error segments is dynamically amplified using an exponential function.

[0023] Furthermore, in step S4, the Adam optimizer is used for gradient descent, and different learning rates are set for the control point position and the stroke width.

[0024] Furthermore, the parsing of coordinate axes and scale information in step S5 includes: using a deep learning object detection model to locate the scale label area near the coordinate axes, and extracting the corresponding numerical text using optical character recognition technology; combining the positions of the outermost horizontal and vertical grid lines obtained through projection analysis in step S2 to determine the coordinate axis positions; and constructing the scale values. Its geometric center coordinates The scale corresponds to the set:

[0025]

[0026] in, This is a set of scale coordinates used to establish a mapping from pixel coordinates to actual numerical values.

[0027] Furthermore, the establishment of the mapping relationship in step S5 adopts linear interpolation mapping: for any pixel coordinate on the vector path The set corresponding to the scale of the corresponding axis Find its corresponding closed interval. The actual value is calculated using the following linear interpolation formula. :

[0028]

[0029] Furthermore, the sampling in step S6 involves generating a series of vertical sampling lines along the X-axis with a user-defined step size or an adaptive step size based on the pixel width. The precise pixel coordinates at any position on the curve are obtained by analytically solving the intersection of the cubic Bézier equation and the vertical sampling lines. .

[0030] Furthermore, the structured data format mentioned in step S6 is CSV data format.

[0031] A system for reverse extraction of line chart data based on differentiable rendering, the system comprising:

[0032] The image acquisition module is used to acquire the line chart image to be processed uploaded by the user;

[0033] The preprocessing module is used to preprocess the line graph image, eliminate interference from non-data elements, and extract the pixel image of a single target data line as the target image.

[0034] The vector path initialization module is used to convert discrete raster pixels in the pixel map extracted by the preprocessing module into an initial vector path composed of a parameterized cubic Bézier curve sequence.

[0035] The differentiable rendering optimization module is used to generate a prediction image through a differentiable renderer by using the control point coordinates and stroke width of the initial vector path obtained by the vector path initialization module as optimizable parameters, calculating the mixture loss function between the prediction image and the target image obtained by the preprocessing module, and backpropagating the gradient to iteratively update the optimizable parameters to obtain the optimized vector path. The mixture loss function is composed of a global loss and an adaptive segmented loss. The adaptive segmented loss divides the path into multiple segments and dynamically increases the weight of high-error segments.

[0036] The coordinate axis parsing and mapping module is used to parse the coordinate axis and scale information of the line graph image and establish a mapping relationship from pixel coordinates to actual values.

[0037] The sampling and data output module is used to sample the vector path optimized by the differentiable rendering optimization module, convert the pixel coordinates of the sampling points into numerical data using the mapping relationship established by the coordinate axis parsing and mapping module, and output them as structured data format.

[0038] This application has the following advantages:

[0039] The method described in this invention supports the reconstruction of raster line graphs into continuous parametric vector curves based on differentiable rendering technology. This method effectively solves the problems of jagged edges and loss of high-frequency details in traditional pixel-based segmentation methods. By constructing an end-to-end reverse engineering process, this invention can achieve sub-pixel-level data restoration in complex scenarios with background noise, text interference, and line intersections, significantly improving the accuracy and robustness of extracting numerical data from scientific charts or financial reports.

[0040] The system possesses high-precision continuous data reconstruction capabilities: Steps S3 and S4 above enable the system to construct a continuous mathematical model using cubic Bézier curves and establish a differentiable relationship between pixel space and vector parameters through a differentiable rendering mechanism. Sub-pixel-level alignment of geometric paths is achieved using a gradient descent algorithm. This overcomes the inherent quantization error in discrete pixel classification, eliminates staircase artifacts, and thus enables the system to possess high-precision continuous data reconstruction capabilities, thereby achieving the invention's intended effect.

[0041] Significantly improves the fitting effect on high-frequency fluctuations and smooth curves: The adaptive piecewise weighted loss function designed in step S4 above supports the system to dynamically perceive local fitting errors, forcing the optimizer to focus on correcting peaks or areas of severe fluctuation that are easily ignored by traditional methods. Combined with the natural smoothness of Bézier curves, this invention can capture smooth trends while preserving high-frequency details, thus enabling the system to have excellent fitting effects when processing line graphs with complex changing trends, thereby achieving the invention's objectives.

[0042] The system exhibits robustness against interference and preservation of topology: Step S2 enables the system to accurately remove text and background noise through multi-feature fusion and morphological filtering. Step S3, based on skeleton extraction, supports the system in maintaining correct geometric connections under complex topologies (such as line intersections and overlaps), avoiding optimization getting trapped in local minima. Combined with the precise analytical sampling in steps S5 and S6, the logical consistency and numerical accuracy of the final output data are ensured, thus achieving the intended effect. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the process of extracting a data table from the original line chart image in a specific embodiment of the present invention.

[0045] Figure 2 This is a flowchart of the preprocessing operation in a specific embodiment of the present invention;

[0046] Figure 3 This is a diagram of the differentiable rendering optimization loop process in a specific embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0048] Example 1:

[0049] This embodiment proposes a reverse extraction method for line graph data based on differentiable rendering, including the following 6 steps: (1) The user uploads the line graph image from which the data to be extracted is obtained; (2) The system preprocesses the line graph image; (3) The system constructs the initial vector path of the target line graph; (4) The system iteratively optimizes the vector path based on the differentiable rendering mechanism; (5) The system parses the coordinate axis and scale information to establish the pixel-to-value mapping relationship; (6) The system samples the optimized vector path and outputs structured numerical data.

[0050] (1) The user uploads a line chart image from which data needs to be extracted;

[0051] Users upload line chart images to the line chart data reverse extraction system. The system supports various line chart images, which may contain one or more data lines, along with non-data elements such as titles, axis labels, and legends. Users can upload images in two formats: .png and .jpeg / .jpg.

[0052] (2) The system preprocesses the line graph image;

[0053] The system performs a series of preprocessing operations on the original line chart image to eliminate the influence of non-data elements and extract clean target data line pixels. The proposed preprocessing method considers two aspects. The first is accuracy, meaning the method should accurately extract clean and continuous target data line pixel information from the original image containing text, grids, and background noise. The second is universality, meaning the method should be applicable to various types of line charts and have good adaptability to line charts with different structures. The preprocessing workflow includes three key stages: text region detection and removal, data region boundary localization, and separation and extraction of individual data lines.

[0054] (a) Text region detection and removal;

[0055] First, the system uses a multi-feature fusion method to identify text elements (such as titles and labels) in the image, combining pixel intensity (capturing dark strokes), high-frequency gradient information (Canny edge detection and dilation), and geometric stability (Maximum Stable Extreme Value Region, MSER) ​​to generate candidate binary masks for text. Next, text regions are further filtered through geometric selection. When a candidate region simultaneously satisfies constraints on edge density, a specific aspect ratio, and connected component complexity, it is determined to be a valid text region. For the determined text boxes, the system employs a Telea-based image inpainting algorithm to fill the background using surrounding gradient information, eliminating interference from the text in subsequent polyline extraction. (See...) Figure 2 (a) Text region detection and removal.

[0056] (b) Data region boundary location;

[0057] After removing non-data line text, the system needs to accurately pinpoint the valid plotting area of ​​the data curve from the line chart image to eliminate interference from legends or edge blanks. This method uses directional morphological filtering and projective contour analysis to locate the actual data line area. Morphological opening operations are performed by applying a "wide and flat" horizontal kernel and a "tall and thin" vertical kernel to enhance the horizontal and vertical border lines, respectively. Subsequently, the cumulative pixel projection in the horizontal and vertical directions is calculated, and the physical boundaries of the chart (axis or grid lines) are determined by detecting peaks, thereby cropping out the valid data line area containing the data curve. See [link to relevant documentation]. Figure 2 (b) Data area boundary location.

[0058] (c) Separation and extraction of a single data line;

[0059] To handle the coexistence of multiple polylines, this method first converts the image from the RGB color space to the more visually uniform CIELAB color space. Next, K-means clustering is used to identify the dominant foreground color in the image, and a deduplication mechanism is combined to generate color prototypes representing each data line. A binary mask is generated based on color similarity, and anisotropic morphological operations are applied to connect broken line segments (such as dashed lines or parts occluded by the grid). Finally, noise is filtered through connected component analysis to extract the pixel image of a single target polyline. Figure 2 (c) Separation and extraction of a single data line.

[0060] (3) The system constructs the initial vector path of the target polyline;

[0061] The method proposes an initialization method based on skeleton extraction, which converts discrete raster pixels into a sequence of Bézier curves with geometric continuity, providing curves with high-quality geometric similarity for subsequent optimization. First, Gaussian smoothing is applied to the single-line pixel image extracted in step (2) to suppress noise. Then, the Canny operator is used to detect edges and extract the maximum connected contour to extract discrete pixels. The method then sorts the set according to the X-axis coordinate to prevent path loops or disorder. Next, it proposes a recursive subdivision algorithm to sort the ordered discrete pixels. The fit is a parameterized sequence of cubic Bézier curves. For a given point set segment, this stage estimates the parameters for each point using chord length parameterization and solves for the control points using linear least squares. Subsequently, the method calculates the maximum Euclidean distance between the fitted curve and the original pixels as the fitting error. If the fitting error Exceeding the preset threshold ( The system splits the point set into two parts at the midpoint of the segment and recursively fits the sub-segments until the error of all segments meets the requirements. Finally, the method uses these discretely generated Bézier curve segments... Connect the beginning and end of the path and reassemble it into a continuous path object, and initialize its control point coordinates and uniform stroke width as the initial input state for the differentiable renderer.

[0062] (4) The system iteratively optimizes the vector path based on a differentiable rendering mechanism;

[0063] The system employs differentiable rendering technology, establishing a bridge between the vector parameter space and the rasterized image space that allows for backpropagation. A pixel-level error loss function is constructed, and the geometric properties of the Bézier curve are iteratively updated using a gradient descent algorithm, achieving a transformation from "approximate fitting" to "precise alignment." (See...) Figure 3 .

[0064] (a) Differentiable rendering and parameter backpropagation;

[0065] The method employs a differentiable rendering architecture based on vector graphics rasterization, using the existing differentiable vector graphics rendering architecture DiffVG as the base renderer, and adapting it for the geometric structure of the line graph data lines. Specifically, the system uses the Bézier curve sequence obtained in step (3) as the vector path input, and represents parameters such as the control point coordinates, pen width, pen color, and transparency of each Bézier segment as optimizable variables, thereby enabling the vector path to be continuously adjusted according to the target image error during the optimization process. It should be noted that this method is not limited to DiffVG, and can also be implemented using other differentiable renderers that support forward rendering and backward gradient propagation of vector paths.

[0066] For the A cubic Bézier curve segment, whose parametric form can be expressed as:

[0067]

[0068] in, This indicates the first segment of the Bézier fragment. There are 1 control point. Let the set of all parameters to be optimized be:

[0069]

[0070] Stroke width is During the forward rendering process, the differentiable renderer adjusts the parameters based on the current parameters. and Render the vector path as a predicted raster image. :

[0071]

[0072] in, This represents a differentiable rendering function. Indicates the stroke color. For any pixel position in the image. The renderer uses the shortest distance from the pixel to the vector path. Estimate its coverage or transparency value to generate the corresponding pixel intensity. This process can be approximated as:

[0073]

[0074]

[0075] in, Indicates pixel coverage. For the Sigmoid function, This is the edge smoothing coefficient. This represents the background pixel value. Using this soft coverage modeling method, pixel intensity can remain differentiable relative to control point coordinates and stroke width.

[0076] Subsequently, the system calculates the predicted image. With target polyline image loss function between And the gradient is backpropagated through the differentiable rendering pipeline:

[0077]

[0078] Therefore, the system can update the Bézier curve control point positions and stroke widths in reverse based on pixel-level errors, gradually aligning the initial vector path towards the target data line. The Adam optimizer can be used for optimization, where the learning rate for control point positions can be set to 0.1–1.0, preferably 0.5; and the learning rate for stroke widths can be set to 0.01–0.2, preferably 0.1. The number of iterations can be set to 20–100, preferably 40, to achieve a balance between optimization accuracy and computational efficiency.

[0079] (b) Adaptive piecewise weighted loss function design;

[0080] A hybrid loss function combining global and segmented loss was designed for pixel calculation of line charts to address the problem that the global mean square error is easily dominated by large-area background and ignores local high-frequency details (such as sharp peaks or small fluctuations).

[0081] First, calculate the global loss. Let the image pixel region be... The predicted image is The target image is Then the global loss is defined as:

[0082]

[0083] This loss is used to constrain the overall position and macroscopic shape of the entire vector path, ensuring that it maintains consistency with the target polyline in its overall trend.

[0084] Secondly, the system calculates the piecewise local loss. For the first... For each Bézier fragment, the system constructs a local region of interest based on the minimum bounding rectangle of the control points of that fragment. and expand outside the bounding box [Number] pixels are used to cover the local error region near the segment. The local error of the segment is defined as:

[0085]

[0086] in, Indicates the first The local region corresponding to each Bézier fragment This represents the local fitting error of the segment. To make the optimizer pay more attention to the poorly fitted local regions, the system further calculates adaptive weights based on the errors of each segment:

[0087]

[0088]

[0089] in, For the total number of Bézier fragments, To prevent stable terms with a denominator of zero, This is the exponential weight scaling factor. Segments with larger errors will receive higher weights, thus obtaining stronger gradient feedback in subsequent optimizations.

[0090] Based on this, the adaptive piecewise loss is defined as:

[0091]

[0092] Finally, the system combines the global loss and the adaptive piecewise loss into a total loss function:

[0093]

[0094] in, These are the piecewise loss weight coefficients, used to control the balance between global shape constraints and local detail constraints. When the size is small, the optimization process focuses more on the overall trend; when When the size is large, the optimization process focuses more on local details.

[0095] In practice, the recursive Bézier fitting error threshold The weight can be set to 0.5–2 pixels, preferably 1 pixel; segmented loss weight coefficient It can be set to 0.1–0.5, preferably 0.25; local region expansion parameter The scaling factor can be set to 3–10 pixels, preferably 5 pixels; exponential weight scaling factor It can be set to 1–5, preferably 2; stability term Can be set to to Preferred By setting the parameters described above, the system can improve the fitting accuracy of peaks, fluctuations, and local bending regions while maintaining the overall stability of the curve topology.

[0096] (5) The system analyzes the coordinate axes and scale information to establish the mapping relationship between pixels and values;

[0097] The method precisely analyzes the coordinate system of the chart and establishes a mapping function from the pixel domain to the data domain to restore the extracted vector geometric paths into data with actual physical meaning. First, the method performs scale text recognition and localization. By integrating a deep learning object detection model (such as ChartDete) to locate the scale label area near the coordinate axis, and using optical character recognition technology (such as EasyOCR) to extract its corresponding numerical text. Next, this stage combines the positions of the outermost horizontal and vertical grid lines obtained through projection analysis in step (2) to determine the true position of the coordinate axis. Furthermore, the method constructs a scale correspondence set and converts the recognized scale values ​​into a corresponding set. Its geometric center coordinates at the center of the image Pair them up to construct corresponding sets of X-axis and Y-axis scales respectively. This process can handle various scenarios, including standard linear axes, inverted axes (X and Y axes interchanged), and partially non-linear scale axes. Finally, the method uses a linear interpolation mapping mechanism to calculate the true values ​​for any pixel coordinates on the vector path. The method is based on the scale set of the corresponding axis. Find its corresponding closed interval. Then, a linear interpolation formula was used. Calculate the corresponding actual value. This piecewise linear mapping strategy not only ensures high accuracy but also accommodates non-uniform scale distributions that may exist in the chart.

[0098] (6) The system samples the optimized vector path and outputs structured numerical data;

[0099] After obtaining the precise coordinate mapping relationship in step (5), the system performs the final data extraction and formatted output. First, the system performs high-precision sampling based on the analytical solution, generating a series of vertical sampling lines along the X-axis with a user-defined step size (or an adaptive step size based on the pixel width). By analytically solving the intersection of the cubic Bézier equation and the vertical lines, the system can obtain the precise pixel coordinates of any position on the curve with pixel-level precision. This effectively avoids the jagged edges and discretization errors caused by rasterization. Subsequently, using the mapping function established in step (5), the pixel coordinates of all intersection points are converted into numerical data with physical meaning (such as time, amount, percentage, etc.). Finally, the system organizes the extracted data sequence into a structured row list and supports exporting it to CSV data format for direct use by downstream data analysis, automatic summarization, or document indexing applications.

[0100] Figure 1The flowchart illustrating the process of extracting a data table from the original line graph image in a specific embodiment of the present invention is as follows: First, the system begins with preprocessing to separate the target line from the original image. Then, the path is rasterized using differentiable rendering, and its parameters (width, control points) are updated by backpropagation loss between the rendered image and the target image. Finally, the optimized vector path is mapped to numerical values ​​using coordinate key information, generating an output CSV file.

[0101] Figure 2 The preprocessing operations in a specific embodiment of the present invention are illustrated in detail. The system generates the target image through three stages: text region detection and removal (a) identifying text noise through multi-feature fusion and removing text noise using the Telea algorithm; data region boundary localization (b) locating the effective chart region by extracting the boundary through the application of directional morphological operations; and separation and extraction of single data lines (c) separating the individual data curve from the background using CIELAB color filtering, K-means clustering, and connectivity component analysis.

[0102] Figure 3 The specific embodiment of the present invention illustrates a differentiable rendering optimization loop process. This process refines the path by minimizing the pixel differences between the rendered output and the target image. A hybrid loss function is employed, combining global loss for overall structure alignment and partial loss (via ROI mask) to enhance local accuracy in complex regions.

[0103] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0104] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for reverse extraction of line chart data based on differentiable rendering, characterized in that, The method includes: S1: Obtain the line chart image to be processed uploaded by the user; S2: Preprocess the line graph image to eliminate interference from non-data elements and extract the pixel image of a single target data line as the target image; S3: Convert the discrete raster pixels in the pixel map extracted in step S2 into an initial vector path composed of a sequence of parameterized cubic Bézier curves. S4: Based on the differentiable rendering mechanism, the control point coordinates and stroke width of the initialized vector path described in step S3 are used as optimizable parameters. A prediction image is generated through a differentiable renderer. The mixed loss function between the prediction image and the target image described in step S2 is calculated, and the gradient is backpropagated to iteratively update the optimizable parameters to obtain the optimized vector path. The mixed loss function is composed of a global loss and an adaptive segmented loss. The adaptive segmented loss divides the path into multiple segments and dynamically increases the weight of high-error segments. S5: Analyze the coordinate axes and scale information of the line graph image to establish a mapping relationship from pixel coordinates to actual values; S6: Sample the vector path optimized in step S4, use the mapping relationship established in step S5 to convert the pixel coordinates of the sampled points into numerical data, and output them as structured data format.

2. The method according to claim 1, characterized in that, The preprocessing described in step S2 includes: S21: Text region detection and removal. First, text regions in the image are identified through multi-feature fusion and geometric screening. Then, the background is filled with a Telea-based image inpainting algorithm to remove the text, resulting in a text-free image. S22: Data region boundary positioning: On the text-free image, directional morphological filtering and projection contour analysis are used to locate the actual data line region and crop out the effective data line region image. S23: Separation and extraction of a single data line: The effective data line region image is converted from RGB color space to CIELAB color space. K-means clustering is used to identify the main foreground color and generate a color prototype. A binary mask is generated based on color similarity. After anisotropic morphological operations and connected component analysis, the pixel image of the single target data line is extracted as the target image.

3. The method according to claim 1, characterized in that, Step S3, converting discrete raster pixels into initial vector paths, includes: applying Gaussian smoothing to the pixel image, using the Canny operator to detect edges and extract the maximum connected contour to obtain an ordered set of discrete pixels. , For the first Coordinates of discrete pixels; A recursive subdivision algorithm is used to estimate the parameters of each point by parameterizing the chord length and then solve for the control points using the linear least squares method. The ordered discrete pixel set is fitted into cubic Bézier curve segments; the fitting error of each segment is calculated. ,when hour, To set a preset threshold, the point set is split into two parts at the midpoint of the segment, and sub-segments are recursively fitted until the error of all segments meets the requirements; the cubic Bézier curve segments are then connected end to end to obtain the initial vector path. , For the first Each Bézier curve segment is initialized with its control point coordinates and uniform stroke width.

4. The method according to claim 1, characterized in that, In step S4, the differentiable renderer, during each iteration of the forward propagation, adjusts the parameters based on the current optimizable parameters. and stroke width Generate predictive raster images The hybrid loss function is: ， in, To predict raster images With the target image The mean square error of the whole image. For adaptive piecewise loss, The ratio of the adaptive piecewise loss function; the adaptive piecewise loss divides the path into... For each Bézier segment, a region of interest is defined based on its control points, and the local error is calculated. The weight of high-error segments is dynamically amplified using an exponential function.

5. The method according to claim 4, characterized in that, In step S4, the Adam optimizer is used for gradient descent, and different learning rates are set for the control point position and the stroke width.

6. The method according to claim 1, characterized in that, Step S5, which involves parsing the coordinate axes and scale information, includes: using a deep learning object detection model to locate the scale label areas near the coordinate axes and extracting the corresponding numerical text using optical character recognition technology; determining the coordinate axis positions by combining the positions of the outermost horizontal and vertical grid lines obtained through projection analysis in step S2; and constructing the scale values. Its geometric center coordinates The scale corresponds to the set: ， in, This is a set of scale coordinates used to establish a mapping from pixel coordinates to actual numerical values.

7. The method according to claim 6, characterized in that, The mapping relationship established in step S5 uses linear interpolation mapping: for any pixel coordinate on the vector path The set corresponding to the scale of the corresponding axis Find the closed interval to which it belongs. The actual value is calculated using the following linear interpolation formula. : 。 8. The method according to claim 1, characterized in that, The sampling described in step S6 involves generating a series of vertical sampling lines along the X-axis with a user-defined step size or an adaptive step size based on the pixel width. The precise pixel coordinates at any position on the curve are obtained by analytically solving the intersection of the cubic Bézier equation and the vertical sampling lines. .

9. The method according to claim 1, characterized in that, The structured data format mentioned in step S6 is CSV data format.

10. A system for reverse extraction of line graph data based on differentiable rendering, characterized in that, The system includes: The image acquisition module is used to acquire the line chart image to be processed uploaded by the user; The preprocessing module is used to preprocess the line graph image, eliminate interference from non-data elements, and extract the pixel image of a single target data line as the target image. The vector path initialization module is used to convert discrete raster pixels in the pixel map extracted by the preprocessing module into an initial vector path composed of a parameterized cubic Bézier curve sequence. The differentiable rendering optimization module is used to generate a prediction image through a differentiable renderer by using the control point coordinates and stroke width of the initial vector path obtained by the vector path initialization module as optimizable parameters, calculating the mixture loss function between the prediction image and the target image obtained by the preprocessing module, and backpropagating the gradient to iteratively update the optimizable parameters to obtain the optimized vector path. The mixture loss function is composed of a global loss and an adaptive segmented loss. The adaptive segmented loss divides the path into multiple segments and dynamically increases the weight of high-error segments. The coordinate axis parsing and mapping module is used to parse the coordinate axis and scale information of the line graph image and establish a mapping relationship from pixel coordinates to actual values. The sampling and data output module is used to sample the vector path optimized by the differentiable rendering optimization module, convert the pixel coordinates of the sampling points into numerical data using the mapping relationship established by the coordinate axis parsing and mapping module, and output them as structured data format.

Images