Center line extraction method and device, electronic equipment and storage medium

Abstract

The invention provides a center line extraction method and device, electronic equipment and a storage medium. The method comprises the steps that a center line image is input into a traditional extraction model for center line extraction, and candidate center points of multiple pixel columns are obtained; based on the candidate central points of the plurality of pixel columns, determining a local image which is not suitable for central line extraction by a traditional extraction model in the central line image; zooming the local image to obtain a target local image, and inputting the target local image into a point reasoning model for point reasoning to obtain candidate points of a plurality of pixel columns; determining a first center point based on the candidate points of the plurality of pixel columns and the candidate center points of the plurality of pixel columns; determining candidate center points of other areas except the area where the target local image is located in the center line image as second center points; the target center line in the center line image is determined based on the first center point and the second center point, the calculation amount of center line extraction is reduced, and the accuracy and real-time performance of center line extraction are improved.

Description

Technical Field

[0001] The present invention relates to the field of laser processing technology, and in particular to a centerline extraction method, device, electronic equipment and storage medium. Background Art

[0002] During laser processing, it's necessary to capture an object's centerline image and then perform centerline extraction on it to help understand the object's shape or outline. Existing centerline extraction methods, for centerline images of varying resolutions and lengths, require scaling a certain number of centerline images to a specified size. These scaled images are then fed into a traditional centerline extraction model to extract the centerline.

[0003] However, the aforementioned centerline extraction methods lose a significant amount of image information during image scaling, resulting in low centerline accuracy. Furthermore, the large number of centerline images fed into traditional centerline models increases the computational complexity exponentially, while computational resources are limited. This results in low real-time performance, making it difficult to meet the real-time requirements of centerline extraction scenarios. Therefore, improving the accuracy and real-time performance of centerline extraction is a pressing issue. Summary of the Invention

[0004] The present invention provides a centerline extraction method, device, electronic equipment and storage medium, which can solve the problems of low accuracy and real-time performance of centerline extraction.

[0005] According to a first aspect of the present invention, a centerline extraction method is provided, the method comprising:

[0006] The centerline image is input into a traditional extraction model to extract the centerline and obtain candidate center points of multiple pixel columns;

[0007] Based on the candidate center points of the plurality of pixel columns, determining in the centerline image a local image that is not suitable for centerline extraction using the traditional extraction model;

[0008] Scaling the local image to obtain a target local image, and inputting the target local image into a point inference model to perform point inference to obtain candidate points of a plurality of pixel columns;

[0009] determining a first center point based on the candidate points of the plurality of pixel columns and the candidate center points of the plurality of pixel columns;

[0010] Determine candidate center points of other areas of the centerline image except the area where the target partial image is located as second center points;

[0011] A target centerline in the centerline image is determined based on the first centerpoint and the second centerpoint.

[0012] According to a second aspect of the present invention, there is provided a centerline extraction device, the device comprising:

[0013] An image input module is used to input the centerline image into a traditional extraction model to extract the centerline and obtain candidate center points of multiple pixel columns;

[0014] a first determining module, configured to determine, in the centerline image, a local image that is not suitable for centerline extraction using the traditional extraction model based on the candidate center points of the plurality of pixel columns;

[0015] A candidate point reasoning module is used to scale the local image to obtain a target local image, and input the target local image into a point reasoning model to perform point reasoning to obtain candidate points of multiple pixel columns;

[0016] a second determining module, configured to determine a first center point based on the candidate points of the plurality of pixel columns and the candidate center points of the plurality of pixel columns;

[0017] a third determining module, configured to determine candidate center points of other areas of the centerline image except the area where the target partial image is located as second center points;

[0018] A fourth determining module is configured to determine a target center line in the center line image based on the first center point and the second center point.

[0019] According to a third aspect of the present invention, there is provided an electronic device comprising a processor and a memory,

[0020] The memory is used to store codes and related data;

[0021] The processor is configured to execute the code in the memory to implement the centerline extraction method as described in any one of the embodiments of the present invention.

[0022] According to a fourth aspect of the present invention, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the centerline extraction method as described in any one of the embodiments of the present invention is implemented.

[0023] In an embodiment of the present invention, a centerline image is input into a conventional extraction model for centerline extraction to obtain candidate center points for multiple pixel columns. Based on the candidate center points for the multiple pixel columns, local images unsuitable for centerline extraction using the conventional extraction model are then determined within the centerline image. The local images are scaled to obtain a target local image, and the target local image is input into a point inference model for point inference to obtain candidate points for multiple pixel columns. A first center point is determined based on the candidate points for the multiple pixel columns and the candidate center points for the multiple pixel columns. The candidate center points of areas of the centerline image other than the area where the target local image is located are determined as second center points. Based on the first and second center points, the target centerline in the centerline image is determined. Specifically, during the centerline extraction process, the centerline image is input into a conventional extraction model for centerline extraction to obtain candidate center points for multiple pixel columns. The candidate center points are determined based on the global features of the centerline image. Local images are then determined based on the candidate center points. The local images are scaled to obtain the target local image, without scaling the entire centerline image to a specified size. The target local image is then input into the point inference model for point inference to obtain candidate points for multiple pixel columns. The candidate points are determined based on the local features of the target local image. The centerline of the centerline image is then extracted based on the candidate points of multiple pixel columns and the candidate center points of multiple pixel columns. The candidate center points are determined based on the global features of the centerline image, and the candidate points are determined based on the local features of the local image. This is equivalent to fusing the local features and global features of centerline images of different resolutions for centerline extraction. In addition, only the local image needs to be scaled to obtain the target local image, which reduces the scaling area of ​​the centerline image, increases the amount of image information in the centerline image, better ensures the accuracy of the original image of the centerline image, and improves the accuracy of centerline extraction. In addition, since only a single centerline image needs to be input into the traditional extraction model (traditional centerline extraction model) for centerline extraction to obtain candidate center points, and a single target local image needs to be input into the point inference model for point inference to obtain candidate points, the number of images input into the model is reduced, which greatly reduces the computational complexity of centerline extraction, improves the real-time performance of centerline extraction, and meets the real-time requirements of the centerline extraction scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 This is a flow chart of a centerline extraction method provided by an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of a training process of a point inference model provided by an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of a training image provided by an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of a denoised image provided by an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of annotated images provided by an embodiment of the present invention;

[0030] Figure 6 is another flow chart of the centerline extraction method provided by an embodiment of the present invention;

[0031] Figure 7 1 is a structural diagram of a centerline extraction device provided by an embodiment of the present invention;

[0032] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatus.

[0035] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0036] Figure 1 This is a flow chart of a centerline extraction method provided by an embodiment of the present invention. This method can be executed by a centerline extraction device, which can be implemented in software and / or hardware. In a specific embodiment, the device can be integrated into an electronic device, such as a computer, a server, etc. The following embodiments will be described using the device integrated into an electronic device as an example. Figure 1 , the method may specifically include the following steps:

[0037] Step 101: Input the centerline image into a traditional extraction model to perform centerline extraction to obtain candidate center points of multiple pixel columns.

[0038] The centerline image can be understood as an image formed by extracting the centerline of an object using a line laser sensor. The traditional extraction model can be understood as a centerline extraction model. A pixel column can be understood as a series of pixels in the vertical direction of the centerline image. A candidate center point can be understood as a point on a pixel column within the centerline extracted by the traditional extraction model.

[0039] Step 102 : Based on the candidate center points of the plurality of pixel columns, a local image that is not suitable for centerline extraction using a traditional extraction model is determined in the centerline image.

[0040] The local image can be understood as an image of a region of the centerline image that is not suitable for centerline extraction using a traditional extraction model. The candidate center points of multiple pixel columns may include coordinates of each candidate center point. The coordinates can be understood as coordinate values ​​on the Y-axis. The Y-axis can be understood as the vertical coordinate axis of the image.

[0041] In an optional embodiment, the number of candidate center points included in each pixel column can be determined; based on the coordinates of the candidate center points of each pixel column, the line width of the laser line of each pixel column can be determined; based on the number of points and the line width, a local image that is not suitable for center line extraction using a traditional extraction model can be determined in the center line image.

[0042] Specifically, determining the line width of the laser line of each pixel column based on the coordinates of the candidate center points of each pixel column can include: determining the maximum coordinate and the minimum coordinate among the candidate center points of each pixel column; determining the coordinate difference between the maximum coordinate and the minimum coordinate of each pixel column as the line width of the laser line of each pixel column. In this way, the line width of the laser line of each pixel column can be calculated more accurately and quickly based on the coordinate difference between the maximum coordinate and the minimum coordinate of the candidate center point of each pixel column, thereby improving the efficiency and accuracy of line width calculation.

[0043] The maximum coordinate can be understood as the maximum Y-axis coordinate value of the candidate center points on the pixel column. The minimum coordinate can be understood as the minimum Y-axis coordinate value of the candidate center points on the pixel column. The laser line can be understood as the line formed by the candidate center points on each pixel column.

[0044] For example, pixel column 1 includes three candidate center points: A, B, and C. These three points continuously form laser line 1. A's coordinate is Y1, B's coordinate is Y2, and C's coordinate is Y3. Y1 < Y2 < Y3. Since Y1 < Y2 < Y3, the maximum coordinate is Y3 and the minimum coordinate is Y1. Therefore, the width of laser line 1 = Y3 - Y1.

[0045] Since the centerline image that is not suitable for centerline extraction by the traditional extraction model contains any of the following areas: halo area, noise area, and centerline area of ​​varying lengths, and the above-mentioned areas have the characteristics of having a point number greater than 1 or a laser line width greater than a specified line width, therefore, based on the point number and line width, determining the local image in the centerline image that is not suitable for centerline extraction by the traditional extraction model can include: when the point number is greater than 1 or the line width is greater than the line width threshold, determining the image in the area where the pixel column is located as the local image that is not suitable for centerline extraction by the traditional extraction model. In this way, based on the point number of the candidate center point and the line width of the laser line, the local image that is not suitable for centerline extraction by the traditional extraction model can be determined more quickly and accurately from the centerline image, thereby improving the accuracy and efficiency of determining the local image, and thereby improving the accuracy of the local image. Among them, the line width threshold can be understood as the minimum value of the preset line width of the laser line.

[0046] For example, if the line width of the laser line 1 is greater than the line width threshold, the image in the area where the pixel column 1 is located may be determined as a local image that is not suitable for centerline extraction using a traditional extraction model.

[0047] Step 103: scaling the local image to obtain a target local image, and inputting the target local image into a point inference model to perform point inference to obtain candidate points of a plurality of pixel columns.

[0048] The point inference model can be understood as a neural network model used to determine the point with the highest probability of being the center point on each pixel column of the local image, as well as the coordinates of the point with the highest probability of being the center point. The point inference model can include a feature extraction network, a regression network, and a classification network. The feature extraction network can be understood as a network used to extract features from the local image to obtain image features. The regression network can be understood as a network used to determine the coordinates of points on the center line of the local image based on the image features of the local image. The regression network can include a pooling layer. In one embodiment, the pooling layer can be a 2*2 layer. In other embodiments, the form of the pooling layer can be adjusted based on actual needs and is not specifically limited here. During the process of determining the coordinates of points on the center line of the local image by the regression network, the pooling layer can adaptively adjust the scaling factor of the feature map based on the feature map corresponding to the image features. The classification network can be understood as a network used to determine the point with the highest probability of being the center point on each pixel column of the local image, as well as the coordinates of the point with the highest probability of being the center point, based on the coordinates of the local image and the points on the center line. The classification network can include a fully connected layer and an activation layer. In one embodiment, the classification network can include two fully connected layers and one activation layer. In other embodiments, the number of fully connected layers and activation layers in the classification network can be adjusted according to actual needs and is not specifically limited here. A candidate point can be understood as a point with the maximum probability of being the center point on the pixel column of the local image.

[0049] In one embodiment, inputting a target partial image into a point inference model for point inference to obtain candidate points for a plurality of pixel columns may include: extracting features from the target partial image using a feature extraction network to obtain image features and sending the image features to a regression network; determining coordinates of points on a center line of the partial image based on the image features of the target partial image using the regression network, and sending the coordinates of the target partial image and the points on the center line to a classification network. Using the classification network, determining, based on the coordinates of the partial image and the points on the center line, a point with a maximum probability of being the center point on each pixel column of the partial image, and determining the coordinates of the point with the maximum probability of being the center point.

[0050] Step 104 : Determine a first center point based on the candidate points of the plurality of pixel columns and the candidate center points of the plurality of pixel columns.

[0051] The candidate points of the plurality of pixel columns may include the probability of each candidate point being a center point, the coordinates of each candidate point, and the grayscale value of each candidate point. The first center point may be understood as the center point on the center line of the local image.

[0052] If the pixel column does not include a candidate point, then the image area is the image area other than the local image in the centerline image, and the candidate center point on the pixel column can be determined as the center point of the image area where the local image is located. If the pixel column includes a candidate point, then the image area is the image area where the local image is located, and the center point of the image area where the local image is located needs to be determined based on the coordinates and grayscale values ​​of the candidate points on the pixel column, as well as the coordinates of the candidate center point. Therefore, in an optional embodiment, when the pixel column does not include a candidate point, the candidate center point included in the pixel column can be determined as the first center point; when the pixel column includes a candidate point, the first center point can be determined based on the grayscale value of each candidate point, the coordinates of each candidate point, and the coordinates of the candidate center point.

[0053] Specifically, since the point inference model is more suitable for determining the center point of a local image than the traditional extraction model, the probability that the candidate point is the first center point is greater than the probability that the candidate point is the first center point. Therefore, when the pixel column includes a candidate point and the grayscale value of the candidate point is greater than the grayscale threshold of the first center point on the center line, the candidate point can be determined to be the first center point. If the pixel column includes both the candidate point and the candidate center point, and the grayscale value of the candidate point is less than or equal to the grayscale threshold of the first center point on the center line, the candidate center point closest to the candidate point on the pixel column can be determined as the first center point. If the pixel column includes the candidate point but does not include the candidate center point, and the grayscale value of the candidate point on the pixel column is less than or equal to the grayscale threshold, it can be determined that the pixel column does not include the first center point. Among them, the grayscale threshold can be understood as the minimum value of the preset grayscale value of the first center point.

[0054] Therefore, determining the first center point based on the grayscale value of each candidate point, the coordinates of each candidate point and the coordinates of the candidate center point may include: when the grayscale value of the candidate point of the pixel column is greater than the grayscale threshold, determining the candidate point as the first center point; when the grayscale value of the candidate point of the pixel column is less than or equal to the grayscale threshold and the pixel column includes the candidate center point, determining the point distance between the candidate point of the pixel column and the candidate center point based on the coordinates of the candidate point of the pixel column and the coordinates of the candidate center point of the pixel column; determining the candidate center point corresponding to the minimum point distance as the first center point; when the grayscale value of the candidate point of the pixel column is less than or equal to the grayscale threshold and the pixel column does not include the candidate center point, determining that the pixel column does not include the first center point. In this way, the first center point can be determined more quickly and accurately based on the grayscale value and coordinates of the candidate point on the pixel column, as well as the coordinates of the candidate center point, thereby improving the accuracy of the first center point.

[0055] Step 105 : Determine the candidate center points of other areas in the centerline image except the area where the target partial image is located as second center points.

[0056] The second center point may be understood as a center point on the center line of other image areas except the local image in the center line image.

[0057] Since the image area other than the local image in the center line image is suitable for input into the traditional extraction model for center line extraction, and the other image areas of the center line image input into the traditional extraction model are not scaled, the image information of the other image areas of the center line image is not lost, thereby improving the accuracy of the candidate center point obtained based on the traditional extraction model. The candidate center point obtained based on the traditional extraction model can be determined as the second center point.

[0058] Step 106 : Determine the target centerline in the centerline image based on the first centerpoint and the second centerpoint.

[0059] In an optional implementation, a plurality of candidate line segments may be obtained by performing straight line fitting on the first center point and the second center point; and the plurality of candidate line segments may be spliced ​​together to obtain the target center line in the center line image.

[0060] Specifically, performing straight line fitting on the first center point and the second center point to obtain multiple candidate line segments may include: fitting the first center point and the second center point using a least squares straight line fitting method to obtain multiple candidate line segments. Splicing multiple candidate line segments to obtain a target center line in a center line image may include: determining the number of line segments of the multiple candidate line segments; when the number of line segments is greater than 1, determining the line lengths of the multiple candidate line segments; sorting the multiple candidate line segments according to the line lengths to obtain multiple first-sorted line segments, the multiple first-sorted line segments including the longest first-sorted line segment; deleting line segments that overlap with the longest first-sorted line segment in the multiple first-sorted line segments, and determining the other line segments in the first-sorted line segments except the longest first-sorted line segment as multiple intermediate line segments; sorting the multiple intermediate line segments to obtain multiple second-sorted line segments, the multiple second-sorted line segments including the longest second-sorted line segment; splicing the longest first-sorted line segment and the longest second-sorted line segment to obtain a target line segment; determining the other line segments in the multiple second-sorted line segments except the longest second-sorted line segment as multiple candidate line segments, triggering the step of determining the number of line segments of the multiple candidate line segments; when the number of line segments is equal to 1, splicing the target line segment and the candidate line segments when the number of line segments is 1 to obtain the target center line.

[0061] Sorting can be understood as sorting line segments according to their length. The first-sorted line segment can be understood as the line segment obtained by sorting multiple candidate line segments. The longest first-sorted line segment can be understood as the longest line segment among the multiple candidate line segments. Multiple intermediate line segments can be understood as the first-sorted line segments excluding the longest first-sorted line segment after deleting the line segments that overlap with the longest first-sorted line segment. The second-sorted line segment can be understood as the line segment obtained by sorting multiple intermediate line segments. The longest second-sorted line segment can be understood as the longest line segment among the second-sorted line segments. The target line segment can be understood as the segment concatenated from the longest first-sorted line segment and the longest second-sorted line segment.

[0062] In an embodiment of the present invention, line segments that overlap with the longest first-sorted line segment among multiple first-sorted line segments are deleted to obtain multiple intermediate line segments. Line segments that do not overlap with the longest first-sorted line segment among the multiple intermediate line segments can be retained. When sorting the multiple intermediate line segments, the influence of the line segments that overlap with the longest first-sorted line segment on the sorting of the multiple intermediate line segments is reduced, thereby improving the sorting accuracy.

[0063] For example, the plurality of candidate line segments include: L1, L2, L3 and L4. The line length of L1 is S1, the line length of L2 is S2, the line length of L3 is S3, and the line length of L4 is S4. S1<S2<S3<S4. L1 and L4 overlap. It can be determined that the number of line segments of the plurality of candidate line segments is 4. When the number of line segments is greater than 1, the line lengths of the plurality of candidate line segments are determined; the plurality of candidate line segments are sorted according to the line lengths to obtain a plurality of first-sorted line segments (L1, L2, L3, L4), and the plurality of first-sorted line segments include the longest first-sorted line segment (L4). The line segment (L1) that overlaps with the longest first-sorted line segment in the plurality of first-sorted line segments is deleted, and the other line segments in the first-sorted line segments except the longest first-sorted line segment are determined as a plurality of intermediate line segments (L2, L3). The plurality of intermediate line segments are sorted to obtain a plurality of second-sorted line segments (L2, L3), wherein the plurality of second-sorted line segments include the longest second-sorted line segment (L3). The longest first-sorted line segment and the longest second-sorted line segment are concatenated to obtain a target line segment (L1'). The remaining line segments of the plurality of second-sorted line segments, excluding the longest second-sorted line segment, are determined as a plurality of candidate line segments (L2). The number of line segments in the plurality of candidate line segments is determined to be equal to 1. The target line segment (L1') and the candidate line segment (L2) when the number of line segments is 1 are concatenated to obtain a target centerline (L').

[0064] In an embodiment of the present invention, during the centerline extraction process, the centerline image is input into a traditional extraction model for centerline extraction to obtain candidate center points of multiple pixel columns, that is, the candidate center points are determined based on the global features of the centerline image. A local image is then determined based on the candidate center points, and the local image is scaled to obtain a target local image. There is no need to scale the entire centerline image to a specified size. The target local image is input into a point inference model for point inference to obtain candidate points of multiple pixel columns, that is, the candidate points are determined based on the local features of the target local image. Centerline extraction of the centerline image is then performed based on the candidate points of the multiple pixel columns and the candidate center points of the multiple pixel columns. The candidate center points are determined based on the global features of the centerline image, and the candidate points are determined based on the local features of the local image. This is equivalent to fusing the local features and global features of centerline images of different resolutions for centerline extraction. Furthermore, only the local image needs to be scaled to obtain the target local image, which reduces the scaling area of ​​the centerline image, increases the amount of image information in the centerline image, better ensures the original image accuracy of the centerline image, and improves the accuracy of centerline extraction. In addition, since only a single centerline image needs to be input into the traditional extraction model (traditional centerline extraction model) for centerline extraction to obtain candidate center points, and a single target local image needs to be input into the point reasoning model for point reasoning to obtain candidate points, the number of images input into the model is reduced, which greatly reduces the computational complexity of centerline extraction, improves the real-time performance of centerline extraction, and meets the real-time requirements of centerline extraction scenarios.

[0065] In some embodiments, the training process of the point inference model may include: Figure 2 Steps shown:

[0066] Step 201: Acquire a training image.

[0067] The training image can be understood as an unprocessed original centerline image.

[0068] Step 202: Preprocess the training image using the image preprocessing model to obtain a preprocessed image.

[0069] The preprocessed image can be understood as an image obtained after denoising the training image.

[0070] In one embodiment, the segmentation model may be used to denoise the original image to obtain a denoised image.

[0071] Step 203 : Using a traditional extraction model, the pre-processed image is annotated to obtain an annotated image. The annotated image includes the center point of the center line and the coordinates of the center point.

[0072] The annotated image can be understood as an image that has been pre-processed and annotated. The annotated content can be the center coordinates of the raised center line in the center line image.

[0073] In one embodiment, a labeling model may be used to label the denoised image to obtain a training image. The labeling model may be a supervised model.

[0074] In the prior art, the annotation model used for image annotation is a semi-supervised model, equivalent to using semi-supervised learning to annotate images. However, semi-supervised learning generally requires a certain degree of manual intervention, resulting in high image annotation costs. In the embodiments of the present invention, however, the annotation model is a supervised model, equivalent to using supervised learning to annotate images, eliminating the need for manual intervention, thus achieving automated image annotation and reducing image annotation costs.

[0075] For example, using the segmentation model Figure 3 The training image shown is denoised to obtain Figure 4 The denoised image shown in the figure is then annotated using the annotation model. Figure 4 The denoised image shown is annotated to obtain Figure 5 The annotated image shown includes the center of the raised centerline. The segmentation model is a supervised model. The annotation model is a supervised model.

[0076] Step 204: Initialize the model parameters of the point inference model.

[0077] Step 205 : Input the annotated image into a point inference model to obtain the coordinates of candidate points that are center points in multiple pixel columns of the annotated image and the probability that the candidate points are center points.

[0078] Here, the probability can be understood as the maximum probability that a point on the pixel column is the center point.

[0079] In an embodiment of the present invention, the annotated image is input into a point inference model to obtain the coordinates of candidate points that are center points in multiple pixel columns of the annotated image and the probability that the candidate points are center points. That is, based on the coordinates on the raised center line in the annotated image, point inference is performed on the points on the pixel column to obtain the candidate point with the greatest probability of being the center point. The image input into the point inference model is a single target local image, which requires less computation and improves the speed at which the point inference model determines the candidate points. Secondly, since the image zoom area is reduced, the image information in the zoomed target local image is richer, which improves the accuracy of the point inference model in determining the candidate points. In addition, since the accuracy and determination speed of the candidate points are high, the accuracy and determination speed of the first center point determined based on the candidate points and the candidate center points are correspondingly improved, thereby improving the accuracy and speed of centerline extraction.

[0080] Step 206: Calculate the loss function value of the point inference model based on the coordinates of the candidate points and the coordinates of the center point.

[0081] Step 207: Use the back propagation algorithm to train the point inference model according to the loss function value until a point inference model that meets the training stop condition is obtained.

[0082] In an embodiment of the present invention, a training image is obtained; the training image is preprocessed using an image preprocessing model to obtain a preprocessed image. The preprocessed image is annotated using a traditional extraction model to obtain an annotated image, wherein the annotated image includes the center point of a centerline and the coordinates of the center point. Model parameters of a point inference model are initialized; the annotated image is input into the point inference model to obtain the coordinates of candidate points that are center points in multiple pixel columns of the annotated image and the probabilities that the candidate points are center points. A loss function value of the point inference model is calculated based on the coordinates of the candidate points and the coordinates of the center point. The point inference model is trained using a backpropagation algorithm based on the loss function value until a point inference model that meets a training stop condition is obtained. This facilitates the input of a single target local image into the point inference model during the centerline extraction process to obtain the coordinates of candidate points that are center points in multiple pixel columns and the probabilities that the candidate points are center points. Specifically, candidate points are determined based on local features of the local image, and the first center point of the local image is determined based on the coordinates and grayscale values ​​of the multiple candidate points and the coordinates of the candidate center point. This reduces the number of target local images, thereby reducing the computational complexity of the point inference model and improving the speed of determining the first center point. In addition, the candidate center point obtained based on the traditional extraction model, that is, the candidate center point is determined based on the global features of the centerline image, and then the candidate center point is determined as the second center point, and finally the target center line is determined according to the first center point and the second center point. This is equivalent to fusing the local features and global features of the centerline image to extract the centerline of the centerline image, which reduces the amount of calculation and improves the accuracy and real-time performance of the centerline extraction.

[0083] The centerline extraction method provided by the embodiment of the present invention is further described below. Figure 6 As shown, Figure 6 FIG. 4 is another flow chart of the centerline extraction method provided by an embodiment of the present invention, which may specifically include the following steps:

[0084] Step 301: Input the centerline image into a traditional extraction model to perform centerline extraction to obtain candidate center points of multiple pixel columns.

[0085] Step 302: Determine the number of candidate center points included in each pixel column.

[0086] Step 303: Determine the maximum coordinate and the minimum coordinate of the candidate center points of each pixel column.

[0087] Step 304 : Determine the coordinate difference between the maximum coordinate and the minimum coordinate of each pixel column as the line width of the laser line of each pixel column.

[0088] Step 305 : When the number of points is greater than 1 or the line width is greater than the line width threshold, the image in the area where the pixel column is located is determined as a local image that is not suitable for centerline extraction using a traditional extraction model.

[0089] Step 306: scaling the local image to obtain a target local image, and inputting the target local image into a point inference model to perform point inference to obtain candidate points of a plurality of pixel columns.

[0090] Step 307 , determining whether the pixel column includes a candidate point, if so, executing step 308 ; if not, executing step 309 .

[0091] Step 308 : Determine the first center point based on the grayscale value of each candidate point, the coordinates of each candidate point, and the coordinates of the candidate center point.

[0092] After executing step 308 , execute step 310 .

[0093] Step 309: Determine the candidate center point included in the pixel column as the first center point.

[0094] Step 310 : Determine the candidate center points of other areas in the centerline image except the area where the target partial image is located as second center points.

[0095] Step 311: Fit the first center point and the second center point using the least squares straight line fitting method to obtain multiple candidate line segments.

[0096] Step 312: splice multiple candidate line segments to obtain the target centerline in the centerline image.

[0097] In an embodiment of the present invention, during the centerline extraction process, the local features and global features of centerline images of different resolutions are fused to perform centerline extraction, and only the local image needs to be scaled to obtain the target local image, thereby reducing the scaling area of ​​the centerline image, increasing the amount of image information in the centerline image, better ensuring the accuracy of the original centerline image, and improving the accuracy of centerline extraction. In addition, since only a single centerline image needs to be input into a traditional extraction model (traditional centerline extraction model) for centerline extraction to obtain a candidate center point, and a single target local image needs to be input into a point inference model for point inference to obtain a candidate point, the number of images input into the model is reduced, thereby greatly reducing the computational complexity of centerline extraction, improving the real-time performance of centerline extraction, and meeting the real-time requirements of the centerline extraction scenario.

[0098] Figure 7 FIG. 1 is a schematic diagram of a centerline extraction device according to an embodiment of the present invention, which is suitable for executing the centerline extraction method according to an embodiment of the present invention. Figure 7 As shown, the device may specifically include:

[0099] An image input module 401 is used to input a centerline image into a traditional extraction model to extract the centerline and obtain candidate center points of multiple pixel columns;

[0100] A first determining module 402 is configured to determine, in the centerline image, a local image that is not suitable for centerline extraction using the traditional extraction model based on the candidate center points of the plurality of pixel columns;

[0101] A candidate point reasoning module 403 is configured to scale the local image to obtain a target local image, and input the target local image into a point reasoning model to perform point reasoning to obtain candidate points of multiple pixel columns;

[0102] A second determining module 404 is configured to determine a first center point based on the candidate points of the plurality of pixel columns and the candidate center points of the plurality of pixel columns;

[0103] A third determining module 405 is configured to determine candidate center points of other areas of the centerline image except the area where the target partial image is located as second center points;

[0104] The fourth determining module 406 is configured to determine the target center line in the center line image based on the first center point and the second center point.

[0105] Optionally, the candidate center points of the plurality of pixel columns include coordinates of each candidate center point, and the first determining module 402 is specifically configured to:

[0106] Determine the number of candidate center points included in each pixel column;

[0107] determining the line width of the laser line of each pixel column according to the coordinates of the candidate center points of each pixel column;

[0108] According to the number of points and the line width, a local image that is not suitable for centerline extraction using the traditional extraction model is determined in the centerline image.

[0109] Optionally, the first determining module 402 determines, based on the number of points and the line width, a local image in the centerline image that is not suitable for centerline extraction using the traditional extraction model, including:

[0110] When the number of points is greater than 1 or the line width is greater than a line width threshold, the image in the area where the pixel column is located is determined as a local image that is not suitable for centerline extraction by the traditional extraction model.

[0111] Optionally, the first determining module 402 determines the line width of the laser line of each pixel column according to the coordinates of the candidate center point of each pixel column, including:

[0112] Determine the maximum and minimum coordinates of the candidate center points of each pixel column;

[0113] The coordinate difference between the maximum coordinate and the minimum coordinate of each pixel column is determined as the line width of the laser line of each pixel column.

[0114] Optionally, the candidate points of the plurality of pixel columns include coordinates of each candidate point and a grayscale value of each candidate point. The second determining module 404 is specifically configured to:

[0115] When the pixel column does not include the candidate point, determining the candidate center point included in the pixel column as the first center point;

[0116] When the pixel column includes the candidate points, the first center point is determined according to the grayscale values ​​of the candidate points, the coordinates of the candidate points, and the coordinates of the candidate center point.

[0117] Optionally, the second determining module 404 determines the first center point according to the grayscale value of each candidate point, the coordinates of each candidate point, and the coordinates of the candidate center point, including:

[0118] When the grayscale value of the candidate point in the pixel column is greater than the grayscale threshold, determining the candidate point as the first center point;

[0119] When the grayscale value of the candidate point of the pixel column is less than or equal to the grayscale threshold and the pixel column includes the candidate center point, determining the point distance between the candidate point of the pixel column and the candidate center point according to the coordinates of the candidate point of the pixel column and the coordinates of the candidate center point of the pixel column;

[0120] Determine the candidate center point corresponding to the minimum point distance as the first center point;

[0121] When the grayscale value of the candidate point of the pixel column is less than or equal to the grayscale threshold and the pixel column does not include the candidate center point, it is determined that the pixel column does not include the first center point.

[0122] Optionally, the fourth determining module 406 is specifically configured to:

[0123] Performing straight line fitting on the first center point and the second center point to obtain multiple candidate line segments;

[0124] The multiple candidate line segments are spliced ​​together to obtain a target centerline in the centerline image.

[0125] Optionally, the fourth determining module 406 performs straight line fitting on the first center point and the second center point to obtain multiple candidate line segments, including:

[0126] The first center point and the second center point are fitted using a least squares straight line fitting method to obtain the multiple candidate line segments.

[0127] Optionally, the fourth determining module 406, which splices the multiple candidate line segments to obtain the target center line in the center line image, includes:

[0128] Determining the number of line segments of the plurality of candidate line segments;

[0129] When the number of line segments is greater than 1, determining the lengths of the plurality of candidate line segments;

[0130] Sorting the plurality of candidate line segments according to the line lengths to obtain a plurality of first sorted line segments, wherein the plurality of first sorted line segments includes the longest first sorted line segment;

[0131] Deleting line segments that overlap with the longest first-sorted line segment from the plurality of first-sorted line segments, and determining other line segments from the first-sorted line segments except the longest first-sorted line segment as a plurality of intermediate line segments;

[0132] Sorting the plurality of intermediate line segments to obtain a plurality of second sorted line segments, wherein the plurality of second sorted line segments includes a longest second sorted line segment;

[0133] splicing the longest first sorted line segment and the longest second sorted line segment to obtain a target line segment;

[0134] Determining the other line segments in the plurality of second-sorted line segments except the longest second-sorted line segment as the plurality of candidate line segments, triggering the step of determining the number of line segments in the plurality of candidate line segments;

[0135] When the number of line segments is equal to 1, the target line segment and the candidate line segment when the number of line segments is 1 are spliced ​​to obtain the target center line.

[0136] Optionally, the training process of the point inference model includes the following steps:

[0137] Get training images;

[0138] Preprocessing the training image using an image preprocessing model to obtain a preprocessed image;

[0139] Annotating the preprocessed image using the traditional extraction model to obtain an annotated image, wherein the annotated image includes a center point of a center line and coordinates of the center point;

[0140] Initializing model parameters of the point reasoning model;

[0141] Inputting the annotated image into the point inference model to obtain coordinates of candidate points that are center points in multiple pixel columns of the annotated image and probabilities that the candidate points are center points;

[0142] Calculating a loss function value of the point reasoning model according to the coordinates of the candidate points and the coordinates of the center point;

[0143] The point inference model is trained according to the loss function value using a back propagation algorithm until a point inference model that meets a training stop condition is obtained.

[0144] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0145] The centerline extraction device provided by the embodiment of the present invention can, during the centerline extraction process, fuse the local features and global features of centerline images of different resolutions to perform centerline extraction, and only needs to scale the local image to obtain the target local image, thereby reducing the scaling area of ​​the centerline image, increasing the amount of image information in the centerline image, better ensuring the accuracy of the original centerline image, and improving the accuracy of centerline extraction. In addition, since only a single centerline image needs to be input into the traditional extraction model (traditional centerline extraction model) for centerline extraction to obtain candidate center points, and a single target local image needs to be input into the point inference model for point inference to obtain candidate points, the number of images input into the model is reduced, which greatly reduces the computational complexity of centerline extraction, improves the real-time performance of centerline extraction, and meets the real-time requirements of centerline extraction scenarios.

[0146] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0147] Please refer to Figure 8 , provides an electronic device 50, including:

[0148] processor 51; and

[0149] a memory 52 for storing executable instructions of the processor;

[0150] The processor 51 is configured to execute the above-mentioned method by executing the executable instructions.

[0151] The processor 51 can communicate with the memory 52 via a bus 53 .

[0152] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned method when executed by a processor.

[0153] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A centerline extraction method, characterized in that: The method comprises: The centerline image is input into a traditional extraction model to extract the centerline and obtain candidate center points of multiple pixel columns; Based on the candidate center points of the plurality of pixel columns, determining in the centerline image a local image that is not suitable for centerline extraction using the traditional extraction model; Scaling the local image to obtain a target local image, and inputting the target local image into a point inference model to perform point inference to obtain candidate points of a plurality of pixel columns; determining a first center point based on the candidate points of the plurality of pixel columns and the candidate center points of the plurality of pixel columns; Determine candidate center points of other areas of the centerline image except the area where the target partial image is located as second center points; A target centerline in the centerline image is determined based on the first centerpoint and the second centerpoint.

2. The method according to claim 1, characterized in that The candidate center points of the plurality of pixel columns include coordinates of each candidate center point, and determining, in the centerline image, a local image that is not suitable for centerline extraction using the traditional extraction model based on the candidate center points of the plurality of pixel columns, includes: Determine the number of candidate center points included in each pixel column; determining the line width of the laser line of each pixel column according to the coordinates of the candidate center points of each pixel column; According to the number of points and the line width, a local image that is not suitable for centerline extraction using the traditional extraction model is determined in the centerline image.

3. The method according to claim 2, characterized in that Determining, in the centerline image, a local image that is not suitable for centerline extraction using the traditional extraction model based on the number of points and the line width includes: When the number of points is greater than 1 or the line width is greater than a line width threshold, the image in the area where the pixel column is located is determined as a local image that is not suitable for centerline extraction by the traditional extraction model.

4. The method according to claim 2, characterized in that The determining the line width of the laser line of each pixel column according to the coordinates of the candidate center point of each pixel column includes: Determine the maximum and minimum coordinates of the candidate center points of each pixel column; The coordinate difference between the maximum coordinate and the minimum coordinate of each pixel column is determined as the line width of the laser line of each pixel column.

5. The method according to claim 2, characterized in that The candidate points of the plurality of pixel columns include coordinates of each candidate point and a grayscale value of each candidate point, and determining the first center point based on the candidate points of the plurality of pixel columns and the candidate center points of the plurality of pixel columns includes: When the pixel column does not include the candidate point, determining the candidate center point included in the pixel column as the first center point; When the pixel column includes the candidate points, the first center point is determined according to the grayscale values ​​of the candidate points, the coordinates of the candidate points, and the coordinates of the candidate center point.

6. The method according to claim 5, characterized in that The determining the first center point according to the grayscale values ​​of the candidate points, the coordinates of the candidate points, and the coordinates of the candidate center point includes: When the grayscale value of the candidate point in the pixel column is greater than the grayscale threshold, determining the candidate point as the first center point; When the grayscale value of the candidate point of the pixel column is less than or equal to the grayscale threshold and the pixel column includes the candidate center point, determining the point distance between the candidate point of the pixel column and the candidate center point according to the coordinates of the candidate point of the pixel column and the coordinates of the candidate center point of the pixel column; Determine the candidate center point corresponding to the minimum point distance as the first center point; When the grayscale value of the candidate point of the pixel column is less than or equal to the grayscale threshold and the pixel column does not include the candidate center point, it is determined that the pixel column does not include the first center point.

7. The method according to claim 1, characterized in that The determining, based on the first center point and the second center point, the target center line in the center line image includes: Performing straight line fitting on the first center point and the second center point to obtain multiple candidate line segments; The multiple candidate line segments are spliced ​​together to obtain a target centerline in the centerline image.

8. The method according to claim 7, characterized in that The performing straight line fitting on the first center point and the second center point to obtain a plurality of candidate line segments includes: The first center point and the second center point are fitted using a least squares straight line fitting method to obtain the multiple candidate line segments.

9. The method according to claim 7, characterized in that The step of splicing the plurality of candidate line segments to obtain the target center line in the center line image includes: Determining the number of line segments of the plurality of candidate line segments; When the number of line segments is greater than 1, determining the lengths of the plurality of candidate line segments; Sorting the plurality of candidate line segments according to the line lengths to obtain a plurality of first sorted line segments, wherein the plurality of first sorted line segments includes the longest first sorted line segment; Deleting line segments that overlap with the longest first-sorted line segment from the plurality of first-sorted line segments, and determining other line segments from the first-sorted line segments except the longest first-sorted line segment as a plurality of intermediate line segments; Sorting the plurality of intermediate line segments to obtain a plurality of second sorted line segments, wherein the plurality of second sorted line segments includes a longest second sorted line segment; splicing the longest first sorted line segment and the longest second sorted line segment to obtain a target line segment; Determining the other line segments in the plurality of second-sorted line segments except the longest second-sorted line segment as the plurality of candidate line segments, triggering the step of determining the number of line segments in the plurality of candidate line segments; When the number of line segments is equal to 1, the target line segment and the candidate line segment when the number of line segments is 1 are spliced ​​to obtain the target center line.

10. The method according to claim 1, characterized in that The training process of the point reasoning model includes the following steps: Get training images; Preprocessing the training image using an image preprocessing model to obtain a preprocessed image; Annotating the preprocessed image using the traditional extraction model to obtain an annotated image, wherein the annotated image includes a center point of a center line and coordinates of the center point; Initializing model parameters of the point reasoning model; Inputting the annotated image into the point inference model to obtain coordinates of candidate points that are center points in multiple pixel columns of the annotated image and probabilities that the candidate points are center points; Calculating a loss function value of the point reasoning model according to the coordinates of the candidate points and the coordinates of the center point; The point inference model is trained according to the loss function value using a back propagation algorithm until a point inference model that meets a training stop condition is obtained.

11. A centerline extraction device, characterized in that: The device comprises: An image input module is used to input the centerline image into a traditional extraction model to extract the centerline and obtain candidate center points of multiple pixel columns; a first determining module, configured to determine, in the centerline image, a local image that is not suitable for centerline extraction using the traditional extraction model based on the candidate center points of the plurality of pixel columns; A candidate point reasoning module is used to scale the local image to obtain a target local image, and input the target local image into a point reasoning model to perform point reasoning to obtain candidate points of multiple pixel columns; a second determining module, configured to determine a first center point based on the candidate points of the plurality of pixel columns and the candidate center points of the plurality of pixel columns; a third determining module, configured to determine candidate center points of other areas of the centerline image except the area where the target partial image is located as second center points; A fourth determining module is configured to determine a target center line in the center line image based on the first center point and the second center point.

12. An electronic device, characterized in that: Including processor and memory, The memory is used to store codes and related data; The processor is used to execute the code in the memory to implement the centerline extraction method according to any one of claims 1 to 10.

13. A storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the centerline extraction method according to any one of claims 1 to 10 is implemented.

Images