A target profile extraction method based on string vibration theory
By using a contour extraction method based on string vibration theory and updating the contour position with a convolutional neural network and a string vibration model, the performance limitations of existing contour models are solved, and higher accuracy target shape extraction is achieved.
Patent Information
- Application Number
- CN202210625740.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-02
- Publication Date
- 2026-02-03
- Estimated Expiration
- 2042-06-02
AI Technical Summary
Existing contour extraction methods mainly rely on dynamic contour models, whose performance is limited by the design of energy terms, resulting in limited ability to extract target boundaries.
A target contour extraction method based on string vibration theory is adopted. Feature contours are extracted through convolutional neural networks, contour vibration models are established using string vibration theory, contour positions are updated through iterative inference, and the learning is encapsulated in an end-to-end network framework.
It improves the accuracy of target shape extraction, avoids the design of energy terms, establishes a robust contour motion model, and has stable progressive evolution capabilities.
Smart Images
Figure CN115294347B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing and deep learning, and particularly relates to a target contour extraction method based on string vibration theory. BACKGROUND
[0002] Compared with the segmentation method of target region extraction, the contour extraction technology has a direct geometric shape expression and can be used for remote sensing mapping, medical lesion detection and other tasks. The existing contour extraction method is mainly a dynamic contour model, which can be divided into two categories of Snake and LevelSet, both of which realize edge capture through the establishment of energy functional, but the performance is greatly limited by the design of energy term, so the current contour model still has limited ability for target boundary extraction. Therefore, we improve it and propose a target contour extraction method based on string vibration theory. SUMMARY
[0003] In order to solve the above technical problems, the present application provides the following technical scheme:
[0004] The target contour extraction method based on string vibration theory comprises the following steps:
[0005] Step 1, the feature contour of the image is extracted through the convolutional neural network, and the coefficient prediction graph alpha, beta of the coefficient prediction is output through the coding-decoding structure;
[0006] Step 2, the initialization position of the feature contour is set;
[0007] Step 3, the corresponding coefficients of the coefficient prediction graph alpha, beta are indexed according to the contour position at the current time to construct the coefficient matrix A1 and the coefficient vector b1 for evolution;
[0008] Step 4, the contour vibration model established according to the string vibration theory is substituted into the contour position at the existing time, the coefficient matrix A1 and the coefficient vector b1 to obtain the updated target contour.
[0009] As a preferred technical scheme of the present application, the step 3 is specifically that the input image is down-sampled and up-sampled by the convolutional neural network with the coding-decoding structure to obtain the coefficient prediction graph alpha, beta with the same size as the input image; and the corresponding coefficients are indexed from the coefficient prediction graph alpha, beta according to the current contour position to obtain the coefficient matrix A1 and the coefficient vector b1.
[0010] As a preferred technical solution of the present application, the method of establishing the contour vibration model in step four according to the string vibration theory is to mine the motion properties of contour evolution itself, establish the motion expression of the contour according to the string vibration theory, discretely express the contour with multiple vertices, derive the iterative reasoning model about the contour evolution, output the prediction result of the target boundary through the end-to-end network framework, and obtain the target contour.
[0011] As a preferred technical solution of the present application, the motion expression of the contour is,
[0012]
[0013] Wherein u is a string, t is a time parameter, x is a space parameter, and a(x), b(x) are coefficients, which represent different positions on the contour when x takes different values.
[0014] As a preferred technical solution of the present application, the algorithm for approximating the continuous contour by the polygon of multiple discrete vertices in step four is to express the contour at time t by N vertices as,
[0015] u t =[u t，1 ,u t，2 ,...,u t,p ,...u t,N ] T Then at the vertex p, there is,
[0016] For p in the expression from 1 to N, there is,
[0017] u t+1 =2u t -u t-1 +Au t (Δt) 2 -b⊙(u t -u t-1 )Δt
[0018] Wherein, is the bitwise multiplication symbol, and the coefficient vector b can be expressed as,
[0019] b=[β1,β2,...,β N ] T ∈R N ,
[0020] Wherein the coefficient matrix A is an N*N three-diagonal matrix, and is expressed as
[0021] The coefficient matrix A1 and the coefficient vector b1 obtained from the coefficients corresponding to the index on the coefficient prediction map a, b are substituted into the coefficient matrix A and the coefficient vector b.
[0022] As a preferred technical solution of the present invention, it further includes step 5, learning the model, which is performed multiple times by repeating steps 3 and 4.
[0023] The beneficial effects of this invention are:
[0024] This target contour extraction method based on string vibration theory proposes a novel contour motion model, which evolves and updates the contour position accordingly. This model effectively detects the target contour and, compared to classical dynamic contour methods, avoids the design of energy terms, establishing a model of contour motion and improving the accuracy of target shape extraction. Compared to existing dynamic contour models, the method in this invention has the following advantages: it has a solid theoretical foundation. Unlike classical dynamic contour models that aim to find the contour that minimizes the energy functional but do not consider the contour motion process, this method not only achieves the equilibrium state of contour evolution through optimization, but the model itself also includes the dynamic motion of the contour; through iterative reasoning of the contour vibration equation, the contour evolves in a progressive manner. Contour evolution and model learning are encapsulated in a closed-loop network framework that can be learned end-to-end, resulting in a simple and stable structure. Attached Figure Description
[0025] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0026] Figure 1 This is a flowchart of a target contour extraction method based on string vibration theory according to the present invention;
[0027] Figure 2 This is a diagram illustrating the contour evolution process of a target contour extraction method based on string vibration theory according to the present invention. Detailed Implementation
[0028] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0029] Example: Figure 1 As shown, the present invention provides a target contour extraction method based on string vibration theory, comprising the following steps:
[0030] Step 1: Extract the feature contours of the image through a convolutional neural network, and output the coefficient prediction maps α and β for coefficient prediction through an encoder-decoder structure;
[0031] Step 2: Set the initial position of the feature contour;
[0032] Step 3: Based on the current contour position, index the corresponding coefficients from the coefficient prediction graphs α and β to construct the coefficient matrix A1 and coefficient vector b1 for evolution;
[0033] Step 4: Based on the contour vibration model established by string vibration theory, substitute the contour position and coefficient matrix A1 and coefficient vector b1 at the existing time to obtain the updated target contour.
[0034] As a preferred technical solution of the present invention, step 3 specifically involves downsampling and upsampling the input image using a convolutional neural network with an encoder-decoder structure to obtain coefficient prediction maps α and β of the same size as the input image; and indexing the corresponding coefficients from the coefficient prediction maps α and β according to the current contour position to obtain the coefficient matrix A1 and the coefficient vector b1.
[0035] The method for establishing the contour vibration model based on string vibration theory in step four is to mine the motion properties of the contour evolution itself, establish the motion expression of the contour based on string vibration theory, and discretize the contour with multiple vertices to derive an iterative inference model for the contour evolution. After passing through an end-to-end network framework, the prediction result of the target boundary is output to obtain the target contour.
[0036] In physics, for a string with characteristics such as flexibility, tautness, and uniformity, the vibration equation of the string under external force is described as follows:
[0037]
[0038] Where u is a chord, t is a time parameter, x is a spatial parameter, and α(x) and β(x) are coefficients, where different values of x represent different positions on the contour. The contour can be viewed as a special type of chord. The coefficients {a, b} in the expression can be rewritten as {α(x), β(x)}, where the coefficients can take different values depending on the change in the spatial parameter x. Therefore, the above expression can be rewritten as follows:
[0039]
[0040] For a continuous contour, it can be approximated by a polygon with multiple discrete vertices. Assume that at time t, the contour is represented by N vertices as u. t =[u t，1 u t，2 , ..., u t，p ,...u t，N ] T According to the definitions of differential and derivative, the expression Each partial differential term in the equation can be approximately represented by the location information of a discrete point, taking vertex p as an example. First, let's consider the internal force term. This reflects the tension on the contour chord, which can be approximated as follows:
[0041]
[0042] Secondly, external force items An approximate expression is as follows:
[0043]
[0044] Finally, the acceleration term. The second derivative with respect to time is approximately expressed as follows:
[0045]
[0046] Substitute all three items into the expression In the process, the vibration model at vertex p on the contour at time t is obtained, which can be described as follows:
[0047]
[0048] For the values of p in the above expression from 1 to N, then we have:
[0049] u t+1 =2u t -u t-1 +Au t (Δt) 2 -b⊙(u t -u t-1 )Δt
[0050] Where ⊙ represents the sign of positional multiplication, the coefficient vector b can be expressed as follows.
[0051] b = [β1, β2, ..., β] N ] T ∈R N ,
[0052] The coefficient matrix A is an N×N tridiagonal matrix, represented as follows:
[0053] Δt is a hyperparameter, such as Figure 2 As shown, given the contour positions at times t and t-1, and the coefficient matrix A1 and coefficient vector b1 predicted by the neural network, which are then substituted into the coefficient matrix A and coefficient vector b, the contour state at the next time t+1 can be inferred. After multiple iterations, the initialized contour gradually converges to the target boundary.
[0054] It also includes step 5, model learning, which involves repeating steps 3 and 4 multiple times to learn the model.
[0055] This target contour extraction method based on string vibration theory proposes a novel contour motion model, which evolves and updates the contour position accordingly. This model effectively detects the target contour and, compared to classical dynamic contour methods, avoids the design of energy terms, establishing a model of contour motion and improving the accuracy of target shape extraction. Compared to existing dynamic contour models, the method in this invention has the following advantages: it has a solid theoretical foundation. Unlike classical dynamic contour models that aim to find the contour that minimizes the energy functional but do not consider the contour motion process, this method not only achieves the equilibrium state of contour evolution through optimization, but the model itself also includes the dynamic motion of the contour; through iterative reasoning of the contour vibration equation, the contour evolves in a progressive manner. Contour evolution and model learning are encapsulated in a closed-loop network framework that can be learned end-to-end, resulting in a simple and stable structure.
[0056] In the above contour evolution, it can be seen that the most crucial elements are the coefficient matrix A and coefficient vector b used by the prediction evolution model to update the contour.
[0057] The coefficient prediction is accomplished using a convolutional neural network with an encoder-decoder structure. In one specific implementation, the encoder module uses DRN-D-22 as the base network and performs three downsampling operations. The decoder network consists of three layers of transposed convolutions to restore resolution. For an input image of size H×W, the output consists of coefficient prediction maps α and β of the same size.
[0058] α(u t，p ), β(u t，p ) represent the vertices u on the contour at time t, respectively. t,p The position is indexed from the coefficient prediction map to find the corresponding coefficient, which is α. p ,β p Based on the position u of the entire contour t This will give us the corresponding coefficient matrix A1 and coefficient vector b1.
[0059] Since it is necessary to directly predict the positions of the vertices of the contour polygon, this invention uses labeled vertices for supervised learning. A key issue is how to pair points from two vertex sets. This invention employs a chamfer distance loss function, described as follows:
[0060]
[0061] in, It represents the set of vertices for prediction and labeling. The expression mainly serves two purposes: vertex matching and distance calculation.
[0062] The coefficients are predicted by a convolutional neural network, and the initial contour is iteratively evolved by a vibration model to obtain the predicted target contour position. Finally, the model is trained in an end-to-end manner and the gradient is backpropagated.
[0063] The contour extraction experimental process of the method of the present invention is as follows:
[0064] This embodiment was performed on the Vaihingen and BingHuts databases. The Vaihingen database was created by the International Society for Photogrammetry and Remote Sensing. This embodiment uses a subset of the original dataset, consisting of 168 images, each 512×512 pixels, with a training / test set split of 100 / 68. The BingHuts database was not manually labeled; the labels were selected from the publicly editable map project OpenStreetMap. It contains 606 images, each 64×64 pixels, with a training / test set split of 335 / 271.
[0065] The contour is defined in advance, according to the expression.
[0066] u t+1 =2u t -u t-1 +Au t (Δt) 2 -b⊙(u t -u t-1 As can be seen from Δt, inferring the contour position at the next moment requires contour information from the two most recent moments, so initialization also requires two contours. This invention initializes using two concentric circles, with an outer circle radius of r and an inner circle radius of 0.9r, sampling N vertices at equal intervals, where N is set to 60. The radius size depends on the relative size of the target in the dataset; for the Vaihingen database, R = 20, and for the BingHuts database, r = 12.
[0067] For convolutional neural networks, the Euclidean distance transform algorithm is used to transform the mask labels into distance labels relative to the boundaries for pre-training. For the input image, the dimensions are all transformed to...
[0068] The size of 256×25∈ is augmented using methods such as flipping, rotating, and scaling, and then normalized. After three downsampling operations, the size changes to 1 / 2, 1 / 4, and 1 / 8 of the initial size. After three upsampling operations, the size is relatively restored to 256x256.
[0069] In this embodiment, the SGD optimizer was used with a decay rate of 1e-5 and a batch size of 10. The learning rate was set to 0.008, and StepLR was used to adjust the learning rate. Each database was trained for 250 epochs, and the learning rate was multiplied by 0.5 every 50 epochs. All experiments were conducted on an NVIDIA 2080Ti GPU.
[0070] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A target contour extraction method based on string vibration theory, characterized in that: Includes the following steps, Step 1: Extract the feature contours of the image through a convolutional neural network, and output the coefficient prediction maps α and β for coefficient prediction through an encoder-decoder structure; Step 2: Set the initial position of the feature contour; Step 3: Based on the current contour position, index the corresponding coefficients from the coefficient prediction graphs α and β to construct the coefficient matrix A1 and coefficient vector b1 for evolution; Step 4: Based on the contour vibration model established by string vibration theory, substitute the contour position and coefficient matrix A1 and coefficient vector b1 at the existing time to obtain the updated target contour. The method for establishing the contour vibration model based on string vibration theory in step 4 involves mining the motion properties of the contour evolution itself, establishing the motion expression of the contour based on string vibration theory, and discretizing the contour using multiple vertices to obtain an iterative inference model for the contour evolution. This model is then processed through an end-to-end network framework to output the predicted result of the target boundary, thus obtaining the target contour. The motion expression of the contour is, Where u is a chord, t is a time parameter, x is a spatial parameter, and α(x) and β(x) are coefficients, where different values of x represent different positions on the contour. The contour vibration model mentioned in step 4 is as follows: in t+1 =2u t -in t-1 +Au t (Δt) 2 -b⊙(u t -in t-1 )Δt Where ⊙ represents the sign of positional multiplication, the coefficient vector b can be expressed as follows. b=[β1,β2,…,β N ] T ∈R N , The coefficient matrix A is an N×N tridiagonal matrix, represented as follows: The coefficient matrix A1 and coefficient vector b1 obtained by indexing the corresponding coefficients from the coefficient prediction graph α, β are substituted into the coefficient matrix A and coefficient vector b.
2. The target contour extraction method based on string vibration theory according to claim 1, characterized in that, Specifically, step 3 involves downsampling and upsampling the input image using a convolutional neural network with an encoder-decoder structure to obtain coefficient prediction maps α and β of the same size as the input image. Based on the current contour position, the corresponding coefficients are indexed from the coefficient prediction map to obtain the coefficient matrix A1 and the coefficient vector b1.
3. The target contour extraction method based on string vibration theory according to claim 1, characterized in that, The algorithm in step four, which approximates a continuous contour using a polygon with multiple discrete vertices, is as follows: at time t, the contour is represented by sampling from N vertices. in t =[in t,1 ,in t,2 ,…,in t,p ,…in t,N ] T , According to the definitions of differential and derivative, the expression Each partial differential term in the equation can be approximately represented by the location information of a discrete point. Taking the vertex p as an example, the first term is the internal force term. This reflects the tension on the contour chord, which can be approximated as follows: Secondly, external force terms An approximate expression is as follows: Finally, the acceleration term. The second derivative with respect to time is approximately expressed as follows: Substitute all three items into the expression In the process, the vibration model at the top p of the profile at time t is obtained, which can be described as follows: For p in this expression to take values from 1 to N, then we have: in t+1 =2u t -in t-1 +Au t (Δt) 2 -b⊙(u t -in t-1 )Δt Where ⊙ represents the sign of positional multiplication, the coefficient vector b can be expressed as follows. b=[β1,β2,…,β N ] T ∈R N , The coefficient matrix A is an N×N tridiagonal matrix, represented as follows: The coefficient matrix A1 and coefficient vector b1 obtained by indexing the corresponding coefficients from the coefficient prediction graph α, β are substituted into the coefficient matrix A and coefficient vector b.
4. The target contour extraction method based on string vibration theory according to claim 1, characterized in that, It also includes step 5, model learning, which involves repeating steps 3 and 4 multiple times to learn the model.