Target region extraction method based on pixel coding

Abstract

The invention discloses a target area extraction method based on pixel coding, and the method comprises the steps: obtaining a vibration image data set of a target object, and dividing the vibration image data set into a training set and a verification set; performing polygon annotation on images in the vibration image data set by using an image annotation tool to generate an annotation file; according to the annotation file, a four-class bidirectional line segment pixel coding method is adopted for coding, and a sparse multi-class mask image is obtained; using the sparse multi-class mask image as a label image, and training the sparse pixel classification network model by using the training set and the verification set to obtain a frozen model; inputting a newly obtained rotor vibration video to be detected into the frozen model for detection, and outputting a sparse multi-category pixel mask graph frame by frame; and decoding the sparse multi-category pixel mask graph frame by frame to obtain a closed polygonal wireframe of the target object. According to the method, extraction of the closed polygon wireframe is realized by utilizing a novel pixel coding mode and through the constructed partitioned sparse pixel classification network model trained in two stages, and efficient and accurate vibration displacement visual measurement of the rotating body is further realized.

Description

Technical Field

[0001] This invention relates to a target region extraction method based on pixel encoding, belonging to the fields of visual vibration displacement measurement and computer vision. Background Technology

[0002] Rotating components are widely used in industry. Minor vibrations are common in industrial rotating machinery, easily leading to equipment failures and affecting equipment operation and personnel safety. Therefore, long-term vibration monitoring of such components is necessary. Vibration monitoring helps in early warning of faults, reducing losses and ensuring production safety. Vision-based non-contact vibration monitoring methods have advantages such as long-distance, non-contact operation, wide applicability, global monitoring, and ease of installation. This method has been successfully applied to the health monitoring of various structures.

[0003] Existing methods primarily utilize detection, segmentation, edge extraction, and phase-based approaches to transform the monitored target in an image into a detection box, target region, target edge region, and local phase representation containing location information. Then, corresponding methods are used to extract displacement or position, followed by subsequent fault analysis. Using detection boxes directly outputs rectangular boxes, which is difficult to adapt to various irregular targets and insensitive to minute movements. Furthermore, significant box jitter between frames easily introduces spurious displacements. Segmentation models obtain pixel-level masks, which can improve displacement extraction accuracy to some extent, but are also prone to missing edge pixels, and model inference is slow and deployment is difficult. Edge detection algorithms typically output edge masks directly, capable of capturing minute jitters at the edges of the monitored target, but perform poorly in complex backgrounds and are extremely sensitive to structural boundary changes. Phase-based methods require multi-scale and multi-directional filters, resulting in relatively slow processing speeds.

[0004] In view of this, the present invention is hereby proposed. Summary of the Invention

[0005] This invention provides a target region extraction method based on pixel encoding, which utilizes a novel pixel encoding method and a sparse pixel classification network model trained in a block-based two-stage manner to extract closed polygon wireframes, and further realizes efficient and accurate visual measurement of vibration displacement of rotating bodies.

[0006] The technical solution of this invention is:

[0007] According to a first aspect of the present invention, a method for extracting target regions based on pixel encoding is provided, comprising:

[0008] Step 1: Obtain the vibration image dataset of the target object and divide it into a training set and a validation set;

[0009] Step 2: Use an image annotation tool to perform polygon annotation on the images in the vibration image dataset to generate annotation files;

[0010] Step 3: Based on the annotation file, use the four-class bidirectional line segment pixel encoding method to encode and obtain a sparse multi-class mask image;

[0011] Step 4: Build a sparse pixel classification network model with two-stage training;

[0012] Step 5: Using sparse multi-class mask images as label images, train the sparse pixel classification network model using the training set and validation set to obtain the frozen model;

[0013] Step 6: Input the newly acquired rotor vibration video to be detected into the frozen model for detection, and output a sparse multi-class pixel mask image frame by frame;

[0014] Step 7: Decode the sparse multi-class pixel mask frame by frame to obtain the closed polygon wireframe of the target object.

[0015] Further, step 3 includes:

[0016] Step 3.1: Construct line segment coding categories based on the four types of bidirectional line segment directions; construct candidate combination directions based on the line segment coding categories; wherein, the candidate combination directions include the first and second candidate combination directions;

[0017] Step 3.2: Extract all annotation line segments from the annotation file to form an annotation line segment set;

[0018] Step 3.3: For the k-th labeled line segment in the labeled line segment set, find the pixel corner point p that is closest to the endpoint of the labeled line segment. k1 ,p k2 Where k = 1, 2, ..., K, K represents the total number of labeled line segments in the labeled line segment set; p k1 The pixel corner closest to the starting point of the labeled line segment; p k2 The pixel corner point closest to the end of the labeled line segment;

[0019] Step 3.4: Based on the polar angle of the labeled line segment, select one candidate combination direction from the first and second candidate combination directions as the selected combination direction for the labeled line segment; for the k-th labeled line segment, from p... k1 Start by truncating the data in pixels based on the maximum pixel length, forming S k Each labeled sub-segment;

[0020] Step 3.6: Generate unidirectional candidate unit line segments towards the endpoint of the labeled line segment in the selected combination direction for the current pixel corner point p; select the best unidirectional candidate unit line segment according to the unit line segment filtering formula;

[0021] Step 3.7: For the j-th labeled sub-segment, take the endpoint of the best unidirectional candidate unit segment of the (j-1)-th labeled sub-segment as the current pixel corner point p of the j-th labeled sub-segment; obtain the best unidirectional candidate unit segment of the j-th labeled sub-segment according to the method in Step 3.6; where j=2,3,...S k ;

[0022] Step 3.8: Based on the line segment encoding category of the best unidirectional candidate unit line segment of all labeled sub-segments of the K labeled line segments, map the best unidirectional candidate unit line segments to generate a sparse multi-class mask image.

[0023] Furthermore, the process involves constructing line segment coding categories based on four types of bidirectional line segment directions, and constructing candidate combination directions based on these line segment coding categories. Specifically:

[0024] Four types of bidirectional line segment directions are defined for pixel nodes: pixel node column direction, pixel node row direction, the first diagonal direction between the top left corner and the bottom right corner of the pixel, and the second diagonal direction between the bottom left corner and the top right corner of the pixel.

[0025] Line segment encoding categories 1, 2, 3, and 4 are constructed for the column direction of pixel nodes, the row direction of pixel nodes, the first diagonal direction between the top left corner and the bottom right corner of a pixel, and the second diagonal direction between the bottom left corner and the top right corner of a pixel.

[0026] The direction with line segment coding category {1,2,4} is selected as the first candidate combination direction, and the direction with line segment coding category {1,2,3} is selected as the second candidate combination direction.

[0027] Furthermore, the step of selecting a candidate combination direction from the first and second candidate combination directions based on the polar angle of the labeled line segment as the selected combination direction for the labeled sub-line segment specifically involves:

[0028] The starting point of the marked line segment is taken as the first pole, and the positive X-axis is taken as the first polar axis. The angle from the first polar axis to the marked line segment, measured counterclockwise from the first polar axis, is called the polar angle.

[0029] The polar angle is divided into four intervals: if it is the first or third interval, the first candidate combination direction is selected as the selected combination direction; if it is the second or fourth interval, the second candidate combination direction is selected as the selected combination direction.

[0030] Further, step 3.8 specifically includes:

[0031] For each best unidirectional candidate line segment, determine the pixel to which the best unidirectional candidate line segment belongs based on the pixel node corresponding to the line segment encoding category; use the line segment encoding category of the best unidirectional candidate line segment as the pixel encoding category of the pixel to which it belongs; construct pixel attributes based on the pixel encoding category and pixel node. ;in, For pixel encoding category, , The pixel node to which the best unidirectional candidate unit line segment belongs;

[0032] If any pixel attribute exists , and Then according to Update the pixel attributes; otherwise, leave the corresponding pixel attributes unchanged. ;

[0033] Pixels with pixel attributes are mapped to sparse multi-class masks, while the remaining background pixels are set to 0, thus generating a sparse multi-class mask image.

[0034] Furthermore, the basis stated Update the pixel attributes as follows:

[0035] when When the combination is first, third, or fifth, then the first, second, and third preset line segment encoding categories are selected as pixel nodes. The pixel encoding category corresponding to the pixel;

[0036] when When it is the second combination, then for pixel nodes The pixel in question is added to the second preset line segment encoding category as the pixel encoding category;

[0037] when When it is the fourth combination, then for pixel nodes The pixel in question is added to the third preset line segment encoding category as the pixel encoding category.

[0038] Furthermore, the sparse pixel classification network model employs a cascaded pre-module and a post-module. The pre-module includes an image block divider, a feature fusion extraction module, a multilayer perceptron, and multiple stacked first Transformer encoding layers connected in sequence. The feature fusion extraction module uses parallel branches: the first branch uses a first convolutional layer, and the second branch uses a statistical feature extractor. The image block divider divides the input original image into a first preset number of pixel blocks. The output of the image block divider serves as the input to the first convolutional layer and the statistical feature extractor. The statistical feature extractor extracts multiple statistical features for each pixel block. These multiple statistical features are concatenated with the features output from the first convolutional layer and then used as the input to the multilayer perceptron. The output of the multilayer perceptron is feature-encoded by stacked first Transformer encoding layers to output a coarse mask. The post-module includes a block selector, an edge attention convolutional module, a second convolutional layer, a second Transformer encoding layer, and a third convolutional layer connected in sequence. The output of the pre-module serves as the input to the post-module. Based on the block selector, edge attention convolutional module, second convolutional layer, multiple stacked second Transformer encoding layers, and third convolutional layer, a coarse mask is output.

[0039] Further, step 7 includes:

[0040] Step 7.1: Build pixel attributes by scanning the sparse multi-class pixel mask image row by row to obtain the pixel encoding categories and pixel nodes. ,in Represents pixel encoding category, , Represents the coordinates of the pixel node;

[0041] Step 7.2: Based on the pixel attributes of the sparse multi-class pixel mask, draw bidirectional line segments for pixels with pixel encoding categories to form a V set of line segments;

[0042] Step 7.3: For each set of line segments, obtain the longest path of that set. ;in, ;

[0043] Step 7.4: For each longest path By unifying the direction of all bidirectional line segments on the path to clockwise, we obtain the longest one-way path. ;

[0044] Step 7.5, based on all Find the smallest bounding rectangle and calculate the coordinates of its center point; take the center point as the second pole and the positive X-axis as the second polar axis;

[0045] Step 7.6: Calculate using the coordinates of the center point of the smallest bounding rectangle. The path angle;

[0046] Step 7.7, for each Construct the angle bisector of the path angle. ; Measured counterclockwise from the second polar axis to the angle bisector The angle is called the path polar angle;

[0047] Step 7.8, for all The path polar angles are sorted according to their magnitude, and the sort number is used as... Numbering; merging adjacent numbers The end point and The starting points are connected by unidirectional line segments in a clockwise direction to obtain a closed polygonal wireframe; where, .

[0048] According to a second aspect of the present invention, a target region extraction apparatus based on pixel encoding is provided, comprising a module of the target region extraction method based on pixel encoding described in any one of the preceding claims.

[0049] According to a third aspect of the present invention, a terminal device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the pixel-encoded target region extraction method described in any of the preceding claims.

[0050] The beneficial effects of this invention are:

[0051] This invention uses a high-speed industrial camera as the image acquisition medium and rotating bodies such as rotors in high-speed video as the objects of vibration displacement measurement, proposing a method for extracting target regions from polygonal edge wireframes. This method encodes edge pixels using multiple line segment types and converts them into sparse pixel multi-class tasks. Specifically, starting from the structural information of the labeled data, this invention proposes a four-class bidirectional line segment pixel encoding method to fit line segments in the continuous space of the labeled file and map them to multi-class pixel labels to construct a sparse multi-class mask image. The sparse pixel classification network model performs multi-class classification on pixels related to the rotating body's edges, outputting a sparse multi-class pixel mask image; and decodes the pixel encoding categories into line segment types, ultimately obtaining a closed polygonal region. Furthermore, the sparse pixel classification network model proposed in this invention is a block-based, two-stage trained sparse pixel classification network model, which can improve the recognition accuracy of local structures and accelerate inference speed. In summary, the method of this invention can achieve polygonal region localization for rotating bodies, exhibiting high accuracy and inference speed, achieving a balance between efficiency and speed in reconstructing the structured boundary of the target region. Furthermore, this invention uses the synchronously acquired eddy current displacement signal as the standard displacement offset and compares the visual displacement measurement results at the same rotational speed. As can be seen from the time-frequency domain diagram and the shaft center trajectory diagram, the encoding and decoding method of this invention, combined with the sparse pixel classification network model, enables the regressed vibration displacement curve to have a high degree of fit with the displacement signal obtained by the eddy current sensor, further verifying the reliability of the method of this invention in measuring the vibration displacement of rotating bodies. Attached Figure Description

[0052] Figure 1 This is a flowchart of the method of the present invention;

[0053] Figure 2 This is a diagram of the rotor of a constant-speed motor on-site.

[0054] Figure 3 This is an example of the best unidirectional candidate unit line segment encoded using the bidirectional line segment pixel encoding method;

[0055] Figure 4 This is an example of mapping the best unidirectional candidate unit line segment;

[0056] Figure 5 This is a schematic diagram of pixel encoding categories provided in the embodiments;

[0057] Figure 6 This is an example of the best unidirectional candidate unit line segment selection;

[0058] Figure 7 This is a diagram of the architecture of a sparse pixel classification network model;

[0059] Figure 8It is the Transformer coding layer structure framework;

[0060] Figure 9 This is the architecture diagram of the edge attention convolution module;

[0061] Figure 10 This is an example of decoding a sparse multi-class pixel mask image;

[0062] Figure 11 It is a comparison of the performance of different architecture algorithms in multi-class classification of sparse pixels of a rotating body;

[0063] Figure 12 This is a performance comparison chart of several verification algorithms with different design architectures;

[0064] Figure 13 This is a comparison chart of the evaluation metrics of various comparison algorithms ((a) is in the y-direction, (b) is in the x-direction);

[0065] Figure 14 These are the time-domain curves in the X direction for different algorithms;

[0066] Figure 15 These are the time-domain curves in the Y direction for different algorithms;

[0067] Figure 16 This is a comparison chart of the center trajectories of different algorithms;

[0068] Figure 17 Here are comparison diagrams of the algorithm's frequency domain ((a) is in the x-direction, (b) is in the y-direction). Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0070] Example 1: As Figures 1-17 As shown, a target region extraction method based on pixel encoding includes:

[0071] Step 1: Obtain the vibration image dataset and divide it into training and validation sets.

[0072] For example: To verify the reliability of the algorithm designed in this invention for measuring the vibration of rotating bodies, a rotor of a constant-speed vibration table was used as the object of experimental image acquisition. A sampling rate was set to construct a constant-speed vibration image dataset A. Non-repeating frames were randomly selected from the constant-speed vibration image dataset as the vibration image dataset. The vibration image dataset was divided into a training set and a validation set, with a random sampling ratio of 0.85:0.15. Different network models were trained and tested using the constant-speed dataset for comparison. Simultaneously, the high-speed camera (Qianyanlang M220M), the light source (LED220W), and the acquisition target were at the same horizontal level. The high-speed camera and the eddy current sensor (0D900800) were used to acquire rotor vibration image data and voltage-displacement reference signals, respectively. Eddy current sensors in both the horizontal (X-direction) and vertical (Y-direction) directions were connected to the data acquisition and analysis instrument, with the probe direction of the eddy current sensor coinciding with the vertical centerline of the rotor. Figure 2 The diagram shows the on-site equipment of the constant-speed motor rotor. Details of the constant-speed vibration image dataset A are shown in Table 1.

[0073] Table 1. Data-related information

[0074]

[0075] Step 2: Use an image annotation tool to perform polygon annotation on the images in the vibration image dataset to generate annotation files.

[0076] For example, the rotating target in the image is labeled with polygons using labelme (with the left vertex of the image as the origin, the vertical downward direction of the left vertex as the positive Y-axis, and the vertical rightward direction of the left vertex as the positive X-axis), and a labeled JSON file is generated. The JSON file contains the start and end point information of the labeled line segments describing the polygonal region of the rotating target.

[0077] Step 3: Based on the annotation file, use the four-class bidirectional line segment pixel encoding method to encode and obtain a sparse multi-class mask image.

[0078] Further, step 3 includes:

[0079] Step 3.1: Construct line segment coding categories based on the four types of bidirectional line segment directions; construct candidate combination directions based on the line segment coding categories; wherein, the candidate combination directions include the first and second candidate combination directions.

[0080] Step 3.2: Extract all annotation line segments from the annotation file to form an annotation line segment set.

[0081] Step 3.3: For the k-th labeled line segment in the labeled line segment set, find the pixel corner point p that is closest to the endpoint of the labeled line segment. k1,p k2 Where k = 1, 2, ..., K, K represents the total number of labeled line segments in the labeled line segment set; p k1 The pixel corner closest to the starting point of the labeled line segment; p k2 The endpoint of the labeled line segment is the pixel corner closest to its endpoint; the endpoints of the labeled line segment are the start and end points. For a pixel, there are four corner points; the top-left corner point is taken as the pixel node, see... Figure 5 .

[0082] Step 3.4: Based on the polar angle of the labeled line segment, select one candidate combination direction from the first and second candidate combination directions as the selected combination direction for the labeled line segment; for the k-th labeled line segment, from p... k1 Start by truncating the data in pixels based on the maximum pixel length, forming S k Each sub-segment is labeled; among them, S k The number of sub-segments in the k-th labeled line segment is equal to the maximum pixel length. The maximum pixel length is determined by: obtaining the nearest pixel corner point to the starting point and the nearest pixel corner point to the ending point of the k-th labeled line segment; subtracting the coordinates of the nearest pixel corner point from the ending point, and selecting the maximum difference between the horizontal and vertical coordinates as the maximum pixel length. (Refer to...) Figure 3 For the first labeled line segment, the pixel corner point p closest to the starting point of the labeled line segment. 11 Mark the nearest pixel corner point p to the end of the line segment. 12 Each pixel is considered to have a length of 1; assuming p 11 The corresponding coordinates are (x11, y11), p 12 The corresponding coordinates are (x12, y12), x12 = x11 - 3, y12 = y11 - 6; therefore, the maximum pixel length = max(|x11 - x12|, |y11 - y12| = 6, that is, the maximum pixel length is 6. For the first labeled line segment from p... 11 The line is first truncated in pixels based on the maximum pixel length, forming 6 labeled sub-segments.

[0083] Step 3.5, p k1 Initialize the current pixel corner point p of the first labeled sub-segment of the k-th labeled line segment.

[0084] Step 3.6: Generate unidirectional candidate unit line segments pointing towards the endpoint of the labeled line segment in the selected combination direction for the current pixel corner point p; select the best unidirectional candidate unit line segment according to the unit line segment filtering formula; for example, such as Figure 3 As shown, p 11 As the current pixel corner point p, for p 11Generate unidirectional candidate unit segments that face the endpoint of the labeled line segment under the selected combination direction (i.e., generate unidirectional candidate unit segments with line segment code categories 1, 2, and 3 that face the endpoint of the labeled line segment); select the best unidirectional candidate unit segment from the three unidirectional candidate unit segments according to the unit segment screening formula, which is the blue line segment in the figure.

[0085] Step 3.7: For the j-th labeled sub-segment, take the endpoint of the best unidirectional candidate unit segment of the (j-1)-th labeled sub-segment as the current pixel corner point p of the j-th labeled sub-segment; obtain the best unidirectional candidate unit segment of the j-th labeled sub-segment according to the method in Step 3.6; where j=2,3,...S k For example: for the second labeled sub-segment, the endpoint of the best unidirectional candidate unit segment of the first labeled sub-segment is taken as the current pixel corner point p of the second labeled sub-segment; according to the method of step 3.6, the best unidirectional candidate unit segment of the second labeled sub-segment is obtained.

[0086] Step 3.8: Based on the line segment encoding category of the best unidirectional candidate unit line segment of all labeled sub-segments of the K labeled line segments, map the best unidirectional candidate unit line segments to generate a sparse multi-class mask image.

[0087] Furthermore, the process involves constructing line segment coding categories based on four types of bidirectional line segment directions, and constructing candidate combination directions based on these line segment coding categories. Specifically:

[0088] Each pixel node defines four types of bidirectional line segment directions: pixel node column direction, pixel node row direction, the first diagonal direction between the top left corner and the bottom right corner of the pixel, and the second diagonal direction between the bottom left corner and the top right corner of the pixel.

[0089] Line segment encoding categories 1, 2, 3, and 4 are constructed for the column direction of pixel nodes, the row direction of pixel nodes, the first diagonal direction between the top left corner and the bottom right corner of a pixel, and the second diagonal direction between the bottom left corner and the top right corner of a pixel.

[0090] The direction with line segment coding category {1,2,4} is selected as the first candidate combination direction, and the direction with line segment coding category {1,2,3} is selected as the second candidate combination direction.

[0091] For example, refer to Figure 5 For a pixel, the top-left corner is the pixel node. The line segment encoding category of the pixel node in the column direction is 1, the line segment encoding category of the pixel node in the row direction is 2, the line segment encoding category of the first diagonal direction between the top-left corner and the bottom-right corner is 3, and the line segment encoding category of the second diagonal direction between the bottom-left corner and the top-right corner is 4.

[0092] Furthermore, the step of selecting a candidate combination direction from the first and second candidate combination directions based on the polar angle of the labeled line segment as the selected combination direction for the labeled sub-line segment specifically involves:

[0093] The starting point of the marked line segment is taken as the first pole, and the positive X-axis is taken as the first polar axis. The angle from the first polar axis to the marked line segment, measured counterclockwise from the first polar axis, is called the polar angle.

[0094] The polar angle is divided into four intervals: if it is the first or third interval, the first candidate combination direction is selected as the selected combination direction; if it is the second or fourth interval, the second candidate combination direction is selected as the selected combination direction.

[0095] For example, the encoding correspondence between polar angle intervals and candidate combination directions is shown in Table 2.

[0096] Table 2. Correspondence between polar angle intervals and candidate combination directions

[0097]

[0098] Furthermore, the unit segment selection formula is as follows:

[0099] ;

[0100] in, This represents the best unidirectional candidate unit line segment; Indicates the first The starting coordinates of a unidirectional candidate unit line segment; No. The coordinates of the endpoint of a unidirectional candidate unit line segment; This indicates the coordinates of the midpoint of the truncated segment of the labeled line within the current pixel. This indicates taking the minimum value.

[0101] For example, refer to Figure 6 The green line segment is the labeled line segment. The maximum pixel length of the labeled line segment is two pixels, that is, two labeled sub-segments. The top-left corner of the labeled line segment is the end point, and the corresponding bottom-right corner is the start point. Based on... Figure 6 From the left half of the diagram, we can see that the polar angle of the marked line segment is in the second interval. Therefore, the second candidate combination direction is selected as the chosen combination direction, which is the direction coded as {1,2,3}. Figure 6 In the middle left half, the blue one-way candidate unit line segments 1, 2, and 3 are one-way candidate unit line segments that are generated by the second labeled sub-line segment and point towards the end of the labeled line segment. The best one-way candidate unit line segment is selected by the unit line segment filtering formula, that is, one-way candidate unit line segment 3 is selected as the best one-way candidate unit line segment.

[0102] Further, step 3.8 specifically includes:

[0103] For each best unidirectional candidate line segment, determine the pixel to which the best unidirectional candidate line segment belongs based on the pixel node corresponding to the line segment encoding category; use the line segment encoding category of the best unidirectional candidate line segment as the pixel encoding category of the pixel to which it belongs; construct pixel attributes based on the pixel encoding category and pixel node. ;in, For pixel encoding category, , The pixel node coordinates to which the best unidirectional candidate unit line segment belongs;

[0104] If any pixel attribute exists , and Then according to Update pixel attributes until "does not exist". , and Otherwise, the corresponding pixel attributes remain unchanged. ;

[0105] The pixel encoding categories with pixel attributes are mapped to sparse multi-class masks (i.e., all pixels with pixel encoding categories of 1, 2, 3, 4 form sparse multi-class masks), and the remaining background pixels are set to 0 to generate sparse multi-class mask images.

[0106] refer to Figure 3 The best unidirectional candidate unit segment obtained for each labeled sub-segment of the labeled line segment is based on... Figure 5 The correspondence between line segment encoding categories and pixel nodes is determined, ultimately identifying the pixel containing the oblique bounding box as the pixel to which the best unidirectional candidate unit line segment belongs; for example... Figure 4 The right half shows the pixel coding category of some of the pixels to which the best unidirectional candidate unit line segments belong (0 represents the pixel coding category of the background pixel, and 1 and 3 represent the pixel coding category of the pixel to which the best unidirectional candidate unit line segments belong).

[0107] Furthermore, the basis stated Update the pixel attributes as follows:

[0108] when When the combination is first, third, or fifth, then the first, second, and third preset line segment encoding categories are selected as pixel nodes. The pixel encoding category corresponding to the pixel; for example, when When it is the first combination, the pixel attribute is updated to , The first preset line segment encoding category; when When it is the third combination, the pixel attribute is updated to , For the second preset line segment encoding category; when When it is the fifth combination, the pixel attribute is updated to , The third preset line segment encoding category;

[0109] when When it is the second combination, then for pixel nodes The pixel is added to the second preset line segment encoding category as the pixel encoding category, and the pixel node... The corresponding pixel attribute is deleted; for example, when When it is the second combination, the pixel attribute is obtained. ;

[0110] when When it is the fourth combination, then for pixel nodes The pixel is added to the third preset line segment encoding category as the pixel encoding category, and the pixel node... The corresponding pixel attribute is deleted. For example, when... When it is the fourth combination, the pixel attribute is obtained. .

[0111] Table 3 shows the repetitive coding error correction table. In the first column, the first combination is (1,2); the second combination is (1,3); the third combination is (1,4); the fourth combination is (2,3); and the fifth combination is (2,4). In the third column, the first preset line segment coding category is 4, which corresponds to the second diagonal direction between the lower left and upper right corners of the pixel; the second preset line segment coding category is 2, which corresponds to the row direction of the pixel node; and the third preset line segment coding category is 1, which corresponds to the column direction of the pixel node. Columns 2 and 4 of the table show the pixel node coordinates before and after error correction.

[0112] Table 3

[0113]

[0114] Step 4: Build a sparse pixel classification network model with two-stage training;

[0115] This model uses the original image as input and sparse multi-class mask images as label images. Unlike dense pixel-level prediction tasks such as UNet, existing networks perform poorly in sparse multi-class pixel prediction tasks. To address this, our method designs a two-stage trained sparse pixel classification network model suitable for this task, which integrates the advantages of high accuracy and fast inference.

[0116] like Figure 7As shown, the sparse pixel classification network model employs a cascaded pre-module and a post-module. The pre-module includes an image block divider, a feature fusion and extraction module, a multilayer perceptron (MLP), and multiple stacked first Transformer encoding layers connected in sequence. The feature fusion and extraction module uses parallel branches: the first branch uses a first convolutional layer, and the second branch uses a statistical feature extractor. The image block divider divides the input original image into a first preset number of pixel blocks. The output of the image block divider serves as the input to the first convolutional layer and the statistical feature extractor. The statistical feature extractor extracts multiple statistical features from each pixel block and combines these statistical features with the features output from the first convolutional layer. After concatenation, the images are used as input to a multilayer perceptron (MLP). The output of the MLP is then processed by a stacked first Transformer encoding layer to produce a coarse mask. The post-processor module consists of a block selector, an edge attention convolutional module, a second convolutional layer, a second Transformer encoding layer, and a third convolutional layer connected in sequence. The output of the pre-processor module is used as input to the post-processor module. Based on the block selector, edge attention convolutional module, second convolutional layer, multiple stacked second Transformer encoding layers, and third convolutional layers, coarse and fine masks are output. The terminal features (i.e., coarse and fine masks) of both the pre-processor and post-processor modules are used to calculate the loss with the label image.

[0117] The edge attention convolution module is used to concatenate the input features after passing them through convolution kernels of different sizes, normalization layers, and activation functions. The concatenated results are then adjusted using a 1×1 convolution kernel to obtain the output features.

[0118] For example, the first convolutional layer uses a series of 3×3 convolutional kernels, 1×1 convolutional kernels, BN layers, and ReLU activation functions; the second convolutional layer uses a series of 3×3 convolutional kernels, 1×1 convolutional kernels, BN layers, and ReLU activation functions; and the third convolutional layer uses a series of 5×5 convolutional kernels, 3×3 convolutional kernels, BN layers, and ReLU activation functions.

[0119] For example, the image blocker in the pre-module is used to divide the original image from 3×512×512 into 1024 3×16×16 pixel blocks; the statistical feature extractor extracts average features, label difference features, difference, Sobel features, and entropy features for each pixel block, and concatenates the extracted "average features, label difference features, difference, Sobel features, and entropy features" with the features output by the first convolutional layer to achieve fusion; the main purpose of the multilayer perceptron (MLP) is to adjust the feature dimension to match the dimension of the input features of the first Transformer encoding layer; the structures of the first and second Transformer encoding layers are as follows: Figure 8As shown, a "stacked" approach is adopted (both the first and second Transformer encoding layers are 10 layers each), with each layer processing the vector output by the previous layer with more refined contextual information. A complete encoding layer consists of two core modules and a standard structure of "residual connections + layer normalization". The "multi-head self-attention mechanism" is the first core of the encoding layer, which allows each element in the sequence to "actively pay attention" to all other elements in the sequence, and integrates the information of these elements according to the degree of attention, ultimately generating a vector containing "global context". The "feedforward network layer" is the second core of the encoding layer, which performs "element-by-element" nonlinear transformation on the contextual features output by the "multi-head self-attention mechanism" through "residual connections + layer normalization", further extracting local features and complex semantic patterns of the sequence. In the post-module, a block selector is first used to select pixel block features containing foreground pixels as edge attention convolutional modules; the structure of the edge attention convolutional module is as follows. Figure 9 As shown, pixels in the target edge region are processed by using 3×3 and 5×5 convolutional kernels. The outputs of the two convolutional kernels of different sizes are concatenated after passing through a normalization layer and an activation function. The concatenated result is then adjusted using a 1×1 convolutional kernel to obtain the output features.

[0120] Considering that outlier line segment types can significantly impact the polygonal bounding boxes of a solid of revolution, this invention designs a Foreground Uncertainty Penalty loss to mitigate this issue. This encourages the model to predict foreground as background when confidence is insufficient, thereby improving overall segmentation accuracy, especially background recognition accuracy. Foreground prediction pixels with a confidence level below a 0.6 threshold are considered uncertain foreground predictions. Subsequently, these uncertain foreground pixels are penalized, forming an additional penalty loss, which is ultimately weighted and added to the main loss.

[0121] The loss function used during the training of the sparse pixel classification network model The expression is:

[0122] ;

[0123]

[0124] in, The main loss function is the cross-entropy loss function. The penalty loss function; The weighting coefficient of the penalty loss function (taken as 1 in this embodiment of the invention) Indicates the number of pixel blocks. Indicates the first The number of pixels in a pixel block; Indicates the first The th pixel block Foreground category prediction value of the nth pixel: Decision variable: If the nth pixel The th pixel block If the predicted foreground category value of a pixel is less than a preset threshold, then Take 1; otherwise, take 0.

[0125] Step 5: Using sparse multi-class mask images as label images, train the sparse pixel classification network model using the training set, and then evaluate the model using the validation set.

[0126] Step 5 specifically involves:

[0127] Step 5.1: All experiments were conducted on a desktop computer equipped with an Intel(R) Core(TM) i7-13700KF CPU @3.60GHz, 32GB of RAM, and a GeForce RTX 4080 GPU (16GB of VRAM). All algorithms used in the experiments employed Torch version 2.7 or higher and CUDA version 12.0 or higher. In this embodiment, the batch size (batch_size) for single image extraction was 1, the input image size was 512×512, the learning rate was 1e-4, the number of iterations was 300, the learning rate decay factor was 1e-4, and all other hyperparameters were set to default values.

[0128] Step 5.2, the specific process of training the sparse pixel classification network model using the training set is as follows:

[0129] The first training phase involves freezing the parameters of the post-modules and updating the parameters of the pre-modules. After multiple training rounds, the best weight file is selected from the candidate weight files as the optimal weight file for the first phase. The second training phase involves freezing the weight parameters of the pre-modules in the loaded optimal weight file from the first phase for the remaining iterations, and updating the parameters of the post-modules until the termination condition is met, obtaining the optimal weight file for the network model. The model with the optimal weight parameters is then used as the trained sparse pixel classification network model; that is, the optimal weight parameters from both phases in the loaded optimal weight file are frozen, resulting in a frozen model. Both training phases use a validation set to evaluate the model, and then the optimal weight file is saved. The specific training process uses the validation set to select the model's weights.

[0130] Step 5.2.1: Call the training script to train the network model;

[0131] Step 5.2.2: Randomly select a batch_size of images from the training set as the current training sample;

[0132] Step 5.2.3: Each image from the training samples described in Step 5.2.2 is sequentially fed into the network model to update the weight parameters. The update of the weight parameters is specifically as follows: the weight parameters are randomly initialized or pre-trained weight parameters are loaded, forward propagation is performed to obtain a set of intermediate parameters, and then the intermediate parameters are used to perform back propagation to update the weight parameters. The new weight parameters will replace the old weight parameters previously used to calculate the forward propagation.

[0133] Step 5.2.4: Perform a forward and backward propagation process on all images in the training set in step 5.2.2 and record it as one training of the network model. Save a weight file for each training. Repeat steps 5.3.2 to 5.3.3 until the number of training iterations of the network model reaches the set number.

[0134] Step 5.2.5: After each training session using the training set, the model output is compared with the true value of the validation set using the validation set, and the evaluation index is calculated. The weight file corresponding to the optimal evaluation index is saved each time.

[0135] Step 6: Input the newly acquired rotor vibration video / test set to be detected into the frozen model for detection, and output sparse multi-class pixel mask image frame by frame;

[0136] Specifically: The newly acquired rotor vibration video / test set to be detected is input into the frozen model for detection. If the target pixel is classified as pixel coding category 1, the output is 1; if the target pixel is classified as pixel coding category 2, the output is 2; if the target pixel is classified as pixel coding category 3, the output is 3; if the target pixel is classified as pixel coding category 4, the output is 4; otherwise, as a background pixel, the output is 0, so as to obtain a sparse multi-class pixel mask map.

[0137] Step 7: Decode the sparse multi-class pixel mask frame by frame to obtain the closed polygon wireframe of the target, and calculate the coordinates of the geometric center point of the closed polygon wireframe; connect the coordinates of the geometric center point of the closed polygon wireframe of the target in all images of the rotor vibration video / test set to obtain the predicted rotational body vibration displacement curve.

[0138] Further, step 7 includes:

[0139] Step 7.1: Perform a row-by-row scan of the sparse multi-class pixel mask image to obtain the pixel encoding categories and pixel nodes (background pixels do not need to be processed), and construct pixel attributes. ,in Represents pixel encoding category , Represents the coordinates of the pixel node.

[0140] Step 7.2: Based on the pixel attributes of the sparse multi-class pixel mask, draw bidirectional line segments for pixels with pixel encoding categories to form a V-group of line segments.

[0141] Step 7.3: Apply a greedy search algorithm to each set of line segments to obtain the longest path in the set of line segments. ;in, , Let v be the longest path in the v-th set of line segments;

[0142] Step 7.4: For each longest path By unifying the direction of all bidirectional line segments on the path to clockwise, we obtain the longest one-way path. , Let v be the longest one-way path in the v-th set of line segments.

[0143] Step 7.5, based on all Find the smallest bounding rectangle and calculate the coordinates of its center point; take the center point as the second pole and the positive X-axis as the second polar axis.

[0144] Step 7.6: Calculate using the coordinates of the center point of the smallest bounding rectangle. The path angle. Specifically, Connecting the starting point with the center point of the smallest bounding rectangle forms the first line segment. The endpoint of the first line segment is connected to the center point of the smallest bounding rectangle to form the second line segment. The angle between the first line segment and the second line segment in a clockwise direction is taken as the path angle.

[0145] Step 7.7, for each Construct the angle bisector of the path angle. ; Measured counterclockwise from the second polar axis to the angle bisector The angle is called the path polar angle;

[0146] Step 7.8, for all The path polar angles are sorted sequentially according to their magnitude, and the sorting number is used as... Numbering; merging adjacent numbers The end point and The starting points are connected by unidirectional line segments in a clockwise direction to obtain a closed polygonal wireframe; where, ;

[0147] Step 7.9: Based on the closed polygon wireframe, solve for the geometric center point of the closed polygon wireframe; use the geometric center point as the position of the target in the image.

[0148] For example, refer to Figure 10 , Figure 10 The right image shows the decoding of the sparse multi-class pixel mask image in the left image. The decoding follows the encoding principles, using the pixel node coordinates and pixel encoding categories obtained through row-by-row scanning to decode into corresponding bi-line segments: For pixels with an encoding category, when the pixel encoding category is 1, a bi-directional line segment is drawn in the column direction of the pixel node; when the pixel encoding category is 2, a bi-directional line segment is drawn in the row direction of the pixel node; when the pixel encoding category is 3, a bi-directional line segment is drawn in the first diagonal direction between the top-left and bottom-right corners of the pixel; when the encoded pixel category is 4, a bi-directional line segment is drawn in the second diagonal direction between the bottom-left and top-right corners of the pixel. For example... Figure 10 As shown, for the sparse multi-class pixel mask in the left image, bidirectional line segments are drawn for pixels with pixel encoding categories based on the pixel attributes of the sparse multi-class pixel mask. Figure 10 The right-hand diagram shows multiple black bidirectional line segments. Because these line segments are not completely connected, they are divided into multiple sets of line segments by breakpoints. Figure 10 All the black bidirectional line segments in the diagram can be divided into three groups based on their break points. Each group consists of multiple bidirectional line segments. Figure 10 A set of line segments is shown in the blue ellipse. A greedy search algorithm is used on each set of line segments to find the longest path. Bidirectional line segments that are not on the longest path are discarded. An example is the set of line segments NO.1, represented by the orange bidirectional line segments. After finding the longest path, all bidirectional line segments are unified to unidirectional (not shown in the diagram) in a clockwise direction. That is, the longest path becomes the longest unidirectional path constructed from clockwise unidirectional line segments. Next, the minimum bounding rectangle is determined based on the highest, lowest, leftmost, and rightmost points of all the longest paths, and the center coordinates of the minimum bounding rectangle are calculated. Then, the included angle is calculated based on the start and end points of the longest path and the center coordinates of the minimum bounding rectangle, and the angle bisectors are obtained. Examples are the green, blue, and orange angle bisectors in the diagram. Finally, the polar angles of each unidirectional longest path are obtained using the angle bisectors, and then sorted to obtain NO.1, NO.2, and NO.3. Finally, connect the start and end points of adjacent longest one-way paths in sequence, that is, connect the end point of the previous path to the start point of the next path with a line segment, as shown in the red line segment in the figure.

[0149] Step 8: Based on the constant speed vibration image dataset and the corresponding equipment, re-acquire data as the rotor vibration video to be detected, and use the displacement signal synchronously acquired by the eddy current sensor as the standard displacement offset (GT) to compare with the rotor vibration displacement curve obtained by the algorithm of this invention, from the time domain diagram, frequency domain diagram and shaft center trajectory of the vibration displacement in the XY direction, respectively. Figure 3 We will compare and verify from several angles.

[0150] I. Based on the constant rotational speed vibration image dataset, the performance of different architecture algorithms is obtained.

[0151] Based on the constant rotational speed vibration image dataset, the four-class bidirectional line segment pixel encoding method of this invention was used to obtain sparse multi-class mask images, and training and validation sets were created. Different architecture models in Table 4 were trained and validated based on the training and validation sets, and the evaluation metrics for pixel classification performance (Precision, Recall, F1) and network inference speed (FPS) were compared. Specific results are shown in Table 4. Figure 12 The architectures involved include: U-Net, which uses an upsampling network encoder-decoder architecture; Depthwise Separable Non-downsampling Net (DSN), which uses a multi-layer depthwise separable convolution stack to preserve the edge structure of the rotation; Pix2Pix, which generates images from images based on conditional generative adversarial networks; ViT-Tiny, which introduces the Transformer architecture to the vision domain for the first time; Swin Transformer V2 (SwinV2-Tiny), which has a stronger normalization mechanism and a friendly hierarchical window attention structure; and ESR-Net, a novel sparse pixel classification network model proposed by the method of this invention.

[0152] Table 4 Performance of models with different architectures

[0153]

[0154] As shown in Table 4, the UNet network, which uses multiple skip connections to achieve detail preservation and multi-scale information fusion, performs excellently in measuring the vibration of rotating bodies. Because UNet performs multiple upsampling and reconstructions of the feature resolution, it can filter pixels around foreground pixels, resulting in decent accuracy. However, the model tends to predict background pixels, leading to almost all pixels related to the edges of the rotating body being predicted as background pixels. Therefore, a large number of pixels constituting the edge structure are missing, such as… Figure 11As shown, the sparse multi-class pixel mask generated by UNet is missing a large number of edge structure pixels. DSN's F1 performance is similar to UNet's; although its precision is worse, its recall is significantly better than UNet, indicating that DSN can find pixels related to the edges of the rotation to some extent, but it is prone to semantic category misclassification. Figure 11 The pixels showing the edge structure generated by DSN are much more complete. Comparing the results of UNet and DSN reveals that appropriate upsampling plays a role in improving accuracy and removing background pixels. In sparse pixel classification, Pix2Pix can directly generate new images, but even with the pix2pixd discriminator, it still lacks the ability to reconstruct fine local structures, and its inference speed is slow and difficult to train, resulting in the worst overall performance. Figure 11 (Not shown in the table). Visual feature extraction backbones based on Transformers (such as ViT) perform particularly well in the visual domain. Their powerful global modeling capabilities can capture long-distance correlations between pixels at the edges of rotating objects, demonstrating superior structure perception. As shown in Table 4, the ViT network outperforms other architectures in all metrics of the sparse pixel multi-classification task. Similarly, SwinV2-Tiny performs well in this task. Compared to ViT, SwinV2-Tiny employs a strategy of progressively downsampling to 1 / 32 of the original feature resolution and effectively avoids the high computational overhead caused by ViT's global self-attention through a sliding window local attention mechanism. Therefore, its pre-inference speed is faster. However, since SwinV2-Tiny still requires an additional upsampling operation after feature extraction, its overall inference speed is basically on par with ViT, showing good performance. Figure 11 As shown, the edge structure pixels generated by ViT and SwinV2 largely maintain the contours of the volume of revolution. Overall, the Transformer architecture outperforms models with convolutional backbones in this task. The proposed network ESR-Net outperforms other architectures in both edge structure pixel classification accuracy and inference speed. Figure 11 The results show that ESR-Net better maintains the continuity between pixels in the multi-class results of the output encoded pixels and exhibits higher accuracy in class discrimination across different regions. These two factors are key to the accuracy of the rotational target region extraction algorithm. Figure 12 It can also be observed that ESR-Net is the best among all evaluation metrics.

[0155] Second, based on the vibration video of the rotor to be tested, the present invention is compared and analyzed with representative models in the field of vibration measurement.

[0156] The vibration video of the rotor to be detected was used as input to different trained models (including the classic segmentation network UNet, the lightweight optimized DeepLabv3+, the edge detection network PIDiNet, and the YOLO series methods widely used in object detection) to obtain vibration displacement results. To comprehensively evaluate the performance of each method, this invention uses the metrics MAE and RMSE to measure the vibration displacement reconstruction accuracy in the time domain, as shown in Table 5. Table 5 shows that the YOLO series networks, due to their output of rectangular boxes, perform poorly in fitting circular contours. Segmentation networks (U-Net, DeepLabv3+, and PIDiNet) generally outperform the YOLO series networks on this dataset. ESR-Net is the best or second best in both MAE and RMSE compared to the segmentation networks, and its FPS is significantly better than UNet. Furthermore, in the Pearson correlation coefficient in the frequency domain, except for PidiNet, which performs poorly in complex backgrounds, and YOLOv10n, which has a large number of missed detections, all performed well, indicating that these methods can accurately capture the frequency domain characteristics of the vibration signal of the rotating body. ESR-Net also performs best or second-best in Pearson. Overall, as... Figure 13 As can be seen, ESR-Net performs best or second best in vibration displacement in both X and Y directions among the four evaluation parameters, maintains a leading position in most indicators, and has a better overall score than the other networks.

[0157] Table 5 Evaluation index results for each model

[0158]

[0159] The quantitative results clearly show that the algorithm of this invention performs best overall in predicting vibration displacement in the XY direction.

[0160] III. Reliability analysis of the performance of the algorithm of this invention in the field of vibration measurement.

[0161] To further verify the reliability of the algorithm of this invention in the field of vibration measurement, the center point coordinates obtained from the six models in Table 5 were regressed using vibration displacement signals. Simultaneously, using the signal obtained from the eddy current sensor as the standard vibration displacement signal (Eddy), the performance of the algorithm of this invention was compared with other algorithms from both the time domain and frequency domain perspectives of the vibration displacement. Figure 14It is evident that the YOLO series performs poorly, and PiDiNet is also unstable. YOLOv10n exhibits numerous missed detections, leading to many abrupt changes after normalization. YOLOv11n, on the other hand, shows inconsistencies when fitting circular outlines to the output rectangles, resulting in numerous spikes in the temporal curve. Unet and DeepLabv3+ perform well, benefiting from the segmentation network's ability to fit targets of arbitrary shapes. The temporal curves of the segmentation network closely match the temporal curves of eddy currents. ESR-Net also performs well in the x-direction. The ESR-Net network fits the target shape using multiple unit pixel line segments, exhibiting a better fitting ability than the pixel-level fitting ability of the segmentation network. Its temporal curve is slightly smoother than that of the segmentation network. Combining the data in the x-direction of Table 5, its RMSE is better than Unet and DeepLabv3+. Similarly, in... Figure 15 In terms of vibration curves in the y-direction, Unet, DeepLabv3+, and ESR-Net networks still perform better, while the YOLO series still suffers from missed detections and floating prediction boxes, and its temporal curves do not fit well.

[0162] In addition, Figure 16 The image compares the rotor's center trajectory diagram, focusing on two main aspects: the degree of fit between the predicted curve and the eddy current curve, and the density of the predicted curves. The former represents the accuracy of the algorithm's prediction in fitting the eddy current, while the latter represents the stability of the algorithm's output. Figure 16 As can be seen, all algorithms perform poorly in fitting the lower half of the curve, while ESR-Net performs best in fitting the upper half, followed by DeepLabv3+. In terms of curve density, ESR-Net performs well, with the smallest trajectory segment width. Its only slight drawback is some unstable regions in the lower left corner, but overall it exhibits the best stability.

[0163] from Figure 17 As can be seen from the frequency domain diagram, apart from the significant differences between PiDiNet and the YOLO series and eddy current signals, the other algorithms are not significantly different. ESR-Net's amplitude difference at the dominant vibration frequencies in the X and Y directions is slightly lower than Unet or DeepLabV3+, and significantly better than the YOLO series. Combined with Table 5, ESR-Net's Pearson performance is also at the optimal or near-optimal level, indicating that ESR-Net's amplitude at the dominant frequency may be slightly worse than the segmentation network, but it performs better at other harmonic frequencies.

[0164] In summary, the YOLO series performed poorly on this dataset because the rectangular bounding boxes it outputs struggle to fit circular outlines. PiDiNet, on the other hand, directly outputs edge pixels, resulting in insufficient overall stability. The segmentation network performed well on this dataset, with its accuracy slightly lower than ESR-Net, but ESR-Net's frame rate was higher. Therefore, ESR-Net exhibited the best overall performance on this dataset.

[0165] Example 2: A target region extraction device based on pixel encoding, comprising: a first module for obtaining a vibration image dataset of the target object and dividing it into a training set and a validation set; a second module for using an image annotation tool to annotate the images in the vibration image dataset with polygons, generating an annotation file; a third module for encoding the images according to the annotation file using a four-class bidirectional line segment pixel encoding method to obtain a sparse multi-class mask image; a fourth module for building a two-stage trained sparse pixel classification network model; a fifth module for using the sparse multi-class mask image as the label image and training the sparse pixel classification network model using the training set and validation set to obtain a frozen model; a sixth module for inputting newly acquired rotor vibration video to be detected into the frozen model for detection and outputting a sparse multi-class pixel mask image frame by frame; and a seventh module for decoding the sparse multi-class pixel mask image frame by frame to obtain the closed polygon wireframe of the target object. For details not described in the above modules, please refer to the relevant description in Example 1.

[0166] Example 3: A terminal device includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the pixel-encoded target region extraction method described in Example 1.

[0167] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A target region extraction method based on pixel encoding, characterized in that, include: Step 1: Obtain the vibration image dataset of the target object and divide it into a training set and a validation set; Step 2: Use an image annotation tool to perform polygon annotation on the images in the vibration image dataset to generate annotation files; Step 3: Based on the annotation file, use the four-class bidirectional line segment pixel encoding method to encode and obtain a sparse multi-class mask image; Step 4: Build a sparse pixel classification network model with two-stage training; Step 5: Using sparse multi-class mask images as label images, train the sparse pixel classification network model using the training set and validation set to obtain the frozen model; Step 6: Input the newly acquired rotor vibration video to be detected into the frozen model for detection, and output a sparse multi-class pixel mask image frame by frame; Step 7: Decode the sparse multi-class pixel mask frame by frame to obtain the closed polygon wireframe of the target object.

2. The target region extraction method based on pixel encoding according to claim 1, characterized in that, Step 3 includes: Step 3.1: Construct line segment coding categories based on the four types of bidirectional line segment directions; construct candidate combination directions based on the line segment coding categories; wherein, the candidate combination directions include the first and second candidate combination directions; Step 3.2: Extract all annotation line segments from the annotation file to form an annotation line segment set; Step 3.3: For the k-th labeled line segment in the labeled line segment set, find the pixel corner point p that is closest to the endpoint of the labeled line segment. k1 ,p k2 Where k = 1, 2, ..., K, K represents the total number of labeled line segments in the labeled line segment set; p k1 The pixel corner closest to the starting point of the labeled line segment; p k2 The pixel corner point closest to the end of the labeled line segment; Step 3.4: Based on the polar angle of the labeled line segment, select one candidate combination direction from the first and second candidate combination directions as the selected combination direction for the labeled line segment; for the k-th labeled line segment, from p... k1 Start by truncating the data in pixels based on the maximum pixel length, forming S k Each labeled sub-segment; Step 3.6: Generate unidirectional candidate unit line segments towards the endpoint of the labeled line segment in the selected combination direction for the current pixel corner point p; select the best unidirectional candidate unit line segment according to the unit line segment filtering formula; Step 3.7: For the j-th labeled sub-segment, take the endpoint of the best unidirectional candidate unit segment of the (j-1)-th labeled sub-segment as the current pixel corner point p of the j-th labeled sub-segment; obtain the best unidirectional candidate unit segment of the j-th labeled sub-segment according to the method in Step 3.6; where j=2,3,...S k ; Step 3.8: Based on the line segment encoding category of the best unidirectional candidate unit line segment of all labeled sub-segments of the K labeled line segments, map the best unidirectional candidate unit line segments to generate a sparse multi-class mask image.

3. The target region extraction method based on pixel encoding according to claim 2, characterized in that, The process involves constructing line segment coding categories based on four types of bidirectional line segment directions, and then constructing candidate combination directions based on these line segment coding categories. Specifically: Four types of bidirectional line segment directions are defined for pixel nodes: pixel node column direction, pixel node row direction, the first diagonal direction between the top left corner and the bottom right corner of the pixel, and the second diagonal direction between the bottom left corner and the top right corner of the pixel. Line segment encoding categories 1, 2, 3, and 4 are constructed for the column direction of pixel nodes, the row direction of pixel nodes, the first diagonal direction between the top left corner and the bottom right corner of a pixel, and the second diagonal direction between the bottom left corner and the top right corner of a pixel. The direction with line segment coding category {1,2,4} is selected as the first candidate combination direction, and the direction with line segment coding category {1,2,3} is selected as the second candidate combination direction.

4. The target region extraction method based on pixel encoding according to claim 2, characterized in that, The step of selecting a candidate combination direction from the first and second candidate combination directions based on the polar angle of the labeled line segment as the selected combination direction for the labeled sub-line segment is as follows: The starting point of the marked line segment is taken as the first pole, and the positive X-axis is taken as the first polar axis. The angle from the first polar axis to the marked line segment, measured counterclockwise from the first polar axis, is called the polar angle. The polar angle is divided into four intervals: if it is the first or third interval, the first candidate combination direction is selected as the selected combination direction; if it is the second or fourth interval, the second candidate combination direction is selected as the selected combination direction.

5. The target region extraction method based on pixel encoding according to claim 2, characterized in that, Step 3.8 specifically includes: For each best unidirectional candidate unit line segment, determine the pixel to which the best unidirectional candidate unit line segment belongs based on the pixel node corresponding to the line segment encoding category; use the line segment encoding category of the best unidirectional candidate unit line segment as the pixel encoding category of the pixel to which it belongs. Pixel attributes are constructed based on pixel encoding category and pixel node. ;in, For pixel encoding category, , The pixel node to which the best unidirectional candidate unit line segment belongs; If any pixel attribute exists , and Then according to Update the pixel attributes; otherwise, leave the corresponding pixel attributes unchanged. ; Pixels with pixel attributes are mapped to sparse multi-class masks, while the remaining background pixels are set to 0, thus generating a sparse multi-class mask image.

6. The target region extraction method based on pixel encoding according to claim 5, characterized in that, The basis Update the pixel attributes as follows: when When the combination is first, third, or fifth, then the first, second, and third preset line segment encoding categories are selected as pixel nodes. The pixel encoding category corresponding to the pixel; when When it is the second combination, then for pixel nodes The pixel in question is added to the second preset line segment encoding category as the pixel encoding category; when When it is the fourth combination, then for pixel nodes The pixel in question is added to the third preset line segment encoding category as the pixel encoding category.

7. The target region extraction method based on pixel encoding according to claim 1, characterized in that, The sparse pixel classification network model adopts a series of pre-modules and post-modules. The pre-module includes an image blocker, a feature fusion and extraction module, a multilayer perceptron, and multiple stacked first Transformer coding layers connected in sequence. The feature fusion and extraction module adopts parallel branches: the first branch adopts a first convolutional layer and the second branch adopts a statistical feature extractor. The image block divider divides the input raw image into a first preset number of pixel blocks. The output of the image block divider serves as the input to the first convolutional layer and the statistical feature extractor. The statistical feature extractor extracts multiple statistical features for each pixel block. The multiple statistical features are concatenated with the features output by the first convolutional layer and then used as the input to the multilayer perceptron. The output of the multilayer perceptron is encoded by stacked first Transformer coding layers to output a coarse mask. The post-module includes a block selector, an edge attention convolutional module, a second convolutional layer, a second Transformer coding layer, and a third convolutional layer connected in sequence. The output of the pre-module serves as the input to the post-module. Based on the block selector, the edge attention convolutional module, the second convolutional layer, and multiple stacked second Transformer coding layers and third convolutional layers, a coarse mask is output.

8. The target region extraction method based on pixel encoding according to claim 1, characterized in that, Step 7 includes: Step 7.1: Build pixel attributes by scanning the sparse multi-class pixel mask image row by row to obtain the pixel encoding categories and pixel nodes. ,in Represents pixel encoding category, , Represents the coordinates of the pixel node; Step 7.2: Based on the pixel attributes of the sparse multi-class pixel mask, draw bidirectional line segments for pixels with pixel encoding categories to form a V set of line segments; Step 7.3: For each set of line segments, obtain the longest path of that set. ;in, ; Step 7.4: For each longest path By unifying the direction of all bidirectional line segments on the path to clockwise, we obtain the longest one-way path. ; Step 7.5, based on all Find the smallest bounding rectangle and calculate the coordinates of its center point; take the center point as the second pole and the positive X-axis as the second polar axis; Step 7.6: Calculate using the coordinates of the center point of the smallest bounding rectangle. The path angle; Step 7.7, for each Construct the angle bisector of the path angle. ; Measured counterclockwise from the second polar axis to the angle bisector The angle is called the path polar angle; Step 7.8, for all The path polar angles are sorted according to their magnitude, and the sort number is used as... Numbering; merging adjacent numbers The end point and The starting points are connected by unidirectional line segments in a clockwise direction to obtain a closed polygonal wireframe; where, .

9. A target region extraction device based on pixel encoding, characterized in that, The module includes the pixel-encoded target region extraction method according to any one of claims 1-8.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the pixel-encoded target region extraction method according to any one of claims 1-8.

Images