Human body contour boundary extraction method in complex scene based on deep learning

Abstract

This invention discloses a method for extracting human contour boundaries in complex scenes based on deep learning, comprising: acquiring the original scene image; processing it using a feature extraction model to obtain a multi-scale feature map, a coarse human contour probability map, and human skeletal information; performing topological analysis based on the feature map and probability map to obtain the contour segments to which limbs belong and occlusion sequence information; performing feature re-aggregation on the concave corner regions in combination with the skeletal information to obtain the concave corner recovery contour segment; inferring the geometric information of the occlusion break endpoints based on the contour segment and obtaining the endpoint geometric constraint parameters; and finally stitching together the above parameters and occlusion sequence information to output a human contour boundary map. This invention overcomes the problems of topological confusion and local geometric blunting under severe occlusion and pose overlap, and achieves human boundary reconstruction with high fidelity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and image processing, and particularly relates to a human body contour boundary extraction method based on deep learning. BACKGROUND

[0002] Human body contour extraction is the basis for machine vision to understand human behavior and posture, and its accuracy directly determines the spatial perception limit of downstream three-dimensional reconstruction, motion capture and human-computer interaction system. In an uncontrolled open scene, accurately stripping the physical boundary line of the human target plays an important bottom supporting role in constructing a high-fidelity digital twin model and realizing fine semantic segmentation.

[0003] Existing contour extraction methods mostly rely on general convolutional neural networks or multi-scale edge detection operators to perform pixel-by-pixel binary classification prediction. When processing an image of a single individual with a clean background, a relatively complete edge response can be output. However, when facing a complex scene, due to the frequent intersection and overlap between limbs and the random occlusion of external irregular objects, the network can only output a highly fragmented probability response map. The feature receptive field of traditional convolution easily mixes the pixel features of different physical surfaces when processing complex intersection regions, resulting in a serious blunting of the output contour at the limb turning points, and producing disordered geometric distortion and breakage in the occluded blind area edge.

[0004] In summary, the existing methods generally face the technical challenges of easy confusion of topological connectivity and easy loss of local geometric features when dealing with highly non-rigid deformation and complex occlusion of the target. Therefore, it is urgent to research a data processing method that can effectively improve the image target boundary reconstruction accuracy and geometric continuity under complex visual working conditions. SUMMARY

[0005] The present application provides a human body contour boundary extraction method based on deep learning in a complex scene, in order to solve the above problems of the prior art.

[0006] The technical scheme provides a human body contour boundary extraction method based on deep learning in a complex scene, comprising:

[0007] Obtaining an original scene image;

[0008] Processing the original scene image by using a pre-trained feature extraction model to obtain a multi-scale feature map set, a human body rough contour probability map and human body skeleton information;

[0009] Performing topological analysis based on the multi-scale feature map set and the human body rough contour probability map to obtain limb attribution contour segments and occlusion layer sequence information;

[0010] Based on human skeletal information and multi-scale feature maps, feature re-aggregation is performed on the concave angle region on the contour segment to which the limb belongs to obtain the concave angle restored contour segment.

[0011] Based on the geometric constraints on the contour segment recovered from the concave corner, the geometric information of the fracture endpoint caused by occlusion is inferred, and the endpoint geometric constraint parameters are obtained.

[0012] Based on the endpoint geometric constraint parameters and occlusion sequence information, the concave corner contour segments are stitched together to output a human body contour boundary map.

[0013] A computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform any one of the above-described methods for extracting human contour boundaries in complex scenes based on deep learning.

[0014] An electronic device, comprising:

[0015] Memory, which stores executable programs;

[0016] A processor is configured to run the program, wherein the program executes the steps described in any one of the above-described methods for extracting human contour boundaries in complex scenes based on deep learning.

[0017] A computer program product includes computer instructions that, when executed by a processor, implement the steps of any one of the methods described above for extracting human contour boundaries in complex scenes based on deep learning.

[0018] Beneficial effects: This invention overcomes the problems of topological confusion and local geometric passivation under severe occlusion and posture overlap, and achieves human body boundary reconstruction with high fidelity. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of a method for extracting human contour boundaries in complex scenes based on deep learning, provided in an embodiment of this application.

[0020] Figure 2 This application provides a schematic diagram of the process of using a pre-trained feature extraction model to process the original scene image to obtain a multi-scale feature map set, a human coarse outline probability map, and human skeleton information.

[0021] Figure 3 This is a schematic diagram of the process for obtaining the contour intersection point provided in the embodiments of this application.

[0022] Figure 4 This is a schematic diagram of the process for locating the concave corner area of ​​the bend provided in the embodiment of this application.

[0023] Figure 5 This is a schematic diagram of the feature re-aggregation process using an anisotropic feature re-aggregation method provided in the embodiments of this application. Detailed Implementation

[0024] Example 1 describes the overall process and basic implementation environment of a method for extracting human contour boundaries in complex scenes based on deep learning. For example... Figure 1 As shown, it includes the following steps:

[0025] Step 101: Obtain the original scene image.

[0026] Specifically, the original scene image is a two-dimensional digital image matrix containing human targets. In complex scenes, human targets in the original scene image are often accompanied by self-occlusion between limbs or local occlusion caused by external objects. Obtaining the original scene image is used to provide a raw data source with a complete pixel distribution for subsequent deep learning networks.

[0027] In some optional implementations, the original scene image can be a video frame captured in real time by an optical camera or a pre-stored static image. Before inputting it into the system, the original scene image can be uniformly scaled to a fixed spatial resolution. In this embodiment, the fixed spatial resolution is set to 512×512 pixels, and the red, green, and blue color channels are standardized to meet the input tensor dimension requirements of the feature extraction model.

[0028] Step 102: Process the original scene image using a pre-trained feature extraction model to obtain a multi-scale feature map set, a coarse outline probability map of the human body, and human skeleton information.

[0029] In this embodiment, the feature extraction model is a deep neural network whose parameters have been pre-optimized on a training set containing a large amount of labeled data. The multi-scale feature map set is a collection of high-dimensional feature tensors extracted by the network encoder at different downsampling levels, with its spatial resolution decreasing progressively while its semantic abstraction increases progressively. The human coarse contour probability map is a single-channel matrix, where the pixel values ​​represent the probability that the spatial location belongs to the outer contour of the human body. The human skeletal information includes the spatial coordinates of each key joint and the connectivity topology between adjacent joints.

[0030] Parallel decoding using a feature extraction model can simultaneously extract the geometric edge probabilities of the lower layers and the anatomical topological priors of the higher layers from a single image, providing multi-dimensional information support for solving complex occlusion problems.

[0031] Step 103: Perform topological analysis based on the multi-scale feature map set and the human body coarse contour probability map to obtain the contour segment to which the limb belongs and the occlusion sequence information.

[0032] Specifically, topological resolution unravels the intersections and overlaps of contour lines caused by limb overlap on a two-dimensional image plane. Limb-assigned contour segments are independent contour segments explicitly assigned to predetermined anatomical locations. Occlusion sequence information is used to define the depth relationships between multiple limb contours within the overlapping region.

[0033] This step is performed because it is difficult to distinguish the direction of intersecting lines by simply relying on edge probabilities. By introducing high-dimensional semantic features from the multi-scale feature map set, the semantic similarity of different branches can be calculated in the feature space, thereby accurately locking the true physical affiliation of each contour branch at the intersection, and inferring the occlusion level by analyzing the local probability distribution features.

[0034] Furthermore, during topological parsing, the coarse human contour probability map can be binarized and thinned to extract skeleton lines with a single pixel width. At the intersection of the skeleton lines, the mean of local features from the multi-scale feature map set is used to perform matching and separation, thereby generating limb-assigned contour segments with clear limb labels.

[0035] Step 104: Based on human skeletal information and multi-scale feature maps, feature re-aggregation is performed on the concave angle regions on the contour segments to which the limbs belong to, to obtain the concave angle restored contour segments.

[0036] Furthermore, based on human skeletal information and multi-scale feature maps, the concave angle regions on the contour segments to which the limbs belong are re-aggregated to obtain the concave angle recovery contour segments and the corrected concave angle vertices on those contour segments.

[0037] In this embodiment, the concave angle region refers to the localized high-curvature contour region formed on the inner side of the limb when a human joint flexes. The concave angle recovery contour segment refers to the contour sequence that has been corrected for coordinates to restore the true physical sharpness.

[0038] In deep learning frameworks, due to the isotropic nature of the receptive field of standard convolutional kernels, when extracting features from concave corner regions, they mix and average pixel features from both sides of the corner with drastically different orientations, resulting in severe geometric blunting of the output contour at the apex. By combining human skeletal information to locate these regions and performing direction-selective feature re-aggregation along the contour, the cross-contamination of features from both sides can be effectively prevented, thereby pulling the off-center vertex coordinates back to the true sub-pixel positions.

[0039] Step 105: Based on the geometric constraints on the contour segment recovered from the concave angle, infer the geometric information of the fracture endpoint caused by occlusion and obtain the endpoint geometric constraint parameters; specifically, obtain the endpoint geometric constraint parameters including the inferred tangent direction angle and the inferred curvature value.

[0040] Specifically, the breakpoint is defined as the endpoint where the human body contour is truncated by a background occluder, causing a sudden drop in signal. The geometric constraint parameters of the endpoint mainly include the tangent direction angle and curvature value at that endpoint.

[0041] In complex occlusion scenarios, the outline of the occluded portion is completely missing. To reasonably fill in the missing part later, it is necessary to extract the curvature evolution trend from the visible outline interval adjacent to the fracture endpoint. Since simple numerical extrapolation is prone to divergence at long distances or to produce abnormal curling that violates common sense of human anatomy, this step uses known high-precision geometric information on the concave corner restored outline segment as anchor points for constraint, thereby obtaining stable endpoint geometric constraint parameters that conform to physical laws.

[0042] Step 106: Based on the endpoint geometric constraint parameters and occlusion sequence information, the concave corner contour segments are stitched together to output the human body contour boundary map.

[0043] In this embodiment, the human body contour boundary map is the final output, a binary graphic or vectorized line set with global consistency.

[0044] Specifically, the system uses the tangent direction and curvature provided in the endpoint geometric constraint parameters as boundary conditions to generate smooth transition connection curves between truncated contour segments belonging to the same limb. Based on the preceding and following relationships determined by the occlusion sequence information, the contours of each limb are sequentially covered and rendered. This step ultimately generates a refined human body outline map with correct topology, accurate joint sharpness, and visually logical occlusion relationships.

[0045] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform any one of the method steps of the deep learning-based method for extracting human contour boundaries in complex scenes proposed in this invention.

[0046] According to another aspect of this application, an electronic device is provided, comprising:

[0047] Memory, which stores executable programs;

[0048] A processor is configured to run the program, wherein the program executes any one of the method steps described in the present invention, which is a method for extracting human contour boundaries in complex scenes based on deep learning.

[0049] According to another aspect of this application, a computer program product is provided, including computer instructions, which, when executed by a processor, implement the steps of any one of the deep learning-based methods for extracting human contour boundaries in complex scenes proposed in this invention.

[0050] In the following embodiments, the specific values ​​for pixel neighborhood radius, number of sampling points, number of histogram intervals, threshold setting, number of network layers and dimensions, attenuation scale parameters, etc., are all preferred implementations verified by the inventors through experiments. Engineers can determine the parameter configuration suitable for specific application scenarios through conventional parameter tuning experiments, based on the actual image resolution, target scale range, and accuracy requirements.

[0051] Example 2 describes the specific network architecture and computation process for extracting multi-level features through a shared encoder and decoding contour and skeletal spatial topology features using a dual-branch structure.

[0052] The original scene image is processed using a pre-trained feature extraction model to obtain a multi-scale feature map set, a coarse human outline probability map, and human skeletal information, such as... Figure 2 As shown, it specifically includes:

[0053] Step 201: Multi-level feature extraction is performed on the original scene image through a shared encoder to obtain a multi-scale feature map set with progressively decreasing spatial resolution.

[0054] Specifically, in this embodiment, a uniform standardized preprocessing operation is performed before the image is input to the encoder. The scale of the original scene image is uniformly scaled to a fixed input size of 512×512 pixels. The scaling process uses bilinear interpolation to maintain sub-pixel-level geometric continuity. Based on this, mean subtraction and standard deviation normalization operations are performed on the red, green, and blue color channels of the scaled image, respectively.

[0055] The shared encoder employs a residual network architecture as its backbone, comprising multiple bottleneck residual unit stages connected in series. It should be understood that the specific hierarchical configuration and channel number settings of the residual network architecture are standard implementations known in the art. During this process, the final outputs of each residual stage are extracted and combined to form a multi-scale feature map set, with its spatial resolution decreasing progressively while its semantic abstraction increases progressively. The multi-scale feature map set is extracted once by the shared encoder and stored in the video memory of the computing device, allowing subsequent decoding branches to access it independently in read-only mode.

[0056] In some alternative implementations, the shared encoder can be replaced by other backbone network architectures with multi-scale feature representation capabilities, such as a feature pyramid network or a visual transformer based on a self-attention mechanism.

[0057] Step 202: Input the multi-scale feature map set into the contour decoding branch, and generate a coarse human contour probability map through step-by-step upsampling and feature concatenation.

[0058] In this embodiment, the contour decoding branch adopts a five-stage skip connection upsampling architecture. This branch uses the deepest feature map with a resolution of 1 / 32 in the multi-scale feature map set as the starting input. Each upsampling module is equipped with a 2x transposed convolutional layer with a kernel size of 3×3 and a stride of 2, which is used to double the spatial resolution of the input features.

[0059] After the output of the transposed convolutional layer, the expanded features are concatenated with feature maps of the same resolution level in the multi-scale feature map set along the channel dimension. This introduces a skip connection mechanism to compensate for the spatial details lost in deep networks. The concatenated feature tensor is then further processed by two 3×3 convolutional layers for channel fusion and feature refinement. Each convolutional layer is followed by batch normalization and a linear rectified activation function.

[0060] Through five levels of progressive upsampling, the spatial resolution of the feature map is successively restored from 1 / 32 to 1 / 1 of the input size. At the end of the network, a 1×1 convolutional layer is used to compress the number of feature channels to a single channel, and an activation function is applied to this single channel. The output is a matrix where each pixel value ranges from 0 to 1, which is the coarse outline probability map of the human body. During the offline training phase, the sum of binary cross-entropy loss and contour region overlap loss is used as the supervision signal for this branch.

[0061] Furthermore, when performing feature concatenation operations in the channel dimension, a channel attention mechanism module can be applied to the feature map derived from the encoder before the concatenation node to assign differentiated response weights to feature channels at different semantic levels and suppress the transmission of background noise features.

[0062] Step 203: Input the multi-scale feature map set into the skeleton decoding branch to generate joint heat map set and limb orientation vector field.

[0063] The skeleton decoding branch adopts the same five-stage skip connection upsampling architecture as the contour decoding branch in terms of network topology. It should be noted that although the two branches share the same feature map input in a read-only manner, they each have independent transposed convolutional layer and channel fusion convolutional layer parameters.

[0064] At the end of this branch, the output layer is divided into two parallel mapping heads. The joint heatmap head produces a 17-channel spatial confidence matrix through a 1×1 convolutional layer; this matrix constitutes the joint heatmap set. These 17 channels correspond to 17 key anatomical joint categories in the human body, and the pixel values ​​in each channel matrix quantitatively represent the confidence level of that physical spatial location as the coordinate of the corresponding joint center.

[0065] Simultaneously, the limb connection vector field head produces a 32-channel vector feature matrix through another 1×1 convolutional layer, which is the limb orientation vector field. This vector field corresponds to 16 predefined limb connection segments, each of which occupies 2 feature channels, used to encode the horizontal and vertical components of the local region, respectively, thereby characterizing the physical orientation of each limb connection segment at the pixel level.

[0066] Optionally, the skeleton decoding branch can also extend outward to a third parallel head, outputting additional channel features corresponding to the scale distribution around the joint, which are used to simultaneously predict the bounding box size of the region of interest that wraps around each physical joint.

[0067] Step 204: Determine joint coordinates based on joint heatmap group, and perform matching on adjacent joints using limb direction vector field to obtain human skeletal information.

[0068] Specifically, the process of extracting specific coordinates from the joint heatmap set relies on local extremum search. For each channel of the joint heatmap set, non-maximum suppression is performed. For example, a circular window of a preset radius is used to slide two-dimensionally across the spatial matrix to extract the two-dimensional coordinates with the highest spatial confidence value in each channel, which are then used as the physical detection result corresponding to that joint category, i.e., the joint coordinates.

[0069] After acquiring all isolated joint coordinates, the anatomical affiliation between adjacent joints needs to be determined. For each pair of predefined physically connected joint types, equally spaced point sampling is performed along the geometric connection between the two detected joint coordinates. At the spatial location of each sampling point, the projection component of the limb orientation vector field along that connection direction is extracted.

[0070] The projected components at all sampling points are numerically summed, and the sum is used as a confidence score to evaluate whether the candidate joint pair has a physical connection. The pair with the highest confidence score is retained as the confirmed connection, and the topology of the confirmed connection is combined with the joint coordinates to form human skeletal information that supports subsequent processing.

[0071] In another specific embodiment, for global joint connection matching, a global joint optimization algorithm based on a bipartite graph matching model can be used to solve the overall optimal solution of the weight cost matrix of all candidate edges, thereby replacing the local greedy matching strategy and reducing the probability of false connections caused by extreme postures.

[0072] Example 3 describes how to solve the problem of contour line mixing caused by limb self-occlusion in the feature space after extracting basic contours and skeletal features, including the complete physical calculation process of intersection point detection, topological assignment, and occlusion sequence inference.

[0073] In one possible implementation, topological analysis is performed based on a multi-scale feature map set and a coarse human contour probability map to obtain the limb-assigned contour segment and occlusion sequence information, such as... Figure 3 As shown, it includes:

[0074] Step 301: Perform skeletonization tracking on the coarse outline probability map of the human body to obtain a set of outline skeleton lines.

[0075] Specifically, the human coarse outline probability map is a continuous probability distribution matrix output by a deep convolutional network. After obtaining this probability map, a binarization operation is applied to it, with a preset probability threshold set to 0.5. Pixels with a probability of not less than 0.5 are marked as foreground contours, and the rest are marked as background.

[0076] An iterative skeletonization algorithm based on morphological thinning is applied to the binary mask, progressively removing edge pixels from the foreground region until all foreground connected regions are reduced to skeleton lines of single-pixel width. During the thinning process, topological connectivity is maintained by determining the local neighborhood state. Eight-connected region tracing is performed on the resulting skeleton map, recording the spatial trajectory of each connected skeleton line as an ordered sequence of pixel coordinates, and compiling them into a set of contour skeleton lines.

[0077] In some alternative implementations, the iterative skeletonization algorithm can be replaced by a distance-transform-based skeleton extraction algorithm, which obtains the skeleton by calculating the Euclidean distance from the foreground pixel to the nearest background pixel and extracting local distance ridges.

[0078] Step 302: Within the pixel neighborhood of the contour skeleton line set, extract the feature gradient direction and feature gradient magnitude of the multi-channel feature map from the multi-scale feature map set.

[0079] After obtaining the skeleton line set, it is necessary to traverse each skeleton line pixel. A circular neighborhood is defined with the currently traversed pixel as the center and a radius of 8 pixels. To balance spatial localization accuracy and semantic discrimination capability, a feature tensor with a spatial resolution of 1 / 4 of the input size is selected from the multi-scale feature map set for extraction. The Sobel edge detection operator is applied to all 256 channels of this feature tensor along the horizontal and vertical directions, and the rate of change of values ​​on the horizontal and vertical axes is calculated. The horizontal and vertical gradient components of each channel are then calculated, and the corresponding feature gradient magnitude is obtained by square root operation. The corresponding feature gradient direction is obtained by calculating the arctangent function.

[0080] Step 303: In order to eliminate the discontinuity of the angle boundary, the feature gradient magnitude is used as the weight to calculate the weighted vector sum of the feature gradient directions of each channel on the unit circle, and the principal gradient direction angle of the corresponding pixel position is determined according to the weighted vector sum.

[0081] Directly averaging the angle values ​​introduces a significant numerical truncation error at the 0-degree and 360-degree boundaries. To eliminate this error, the feature gradient directions of each channel are mapped to unit vectors on the unit circle, and a physically weighted sum is performed:

[0082] S _x (x, y) = Σ(m _c (x, y)*cos(θ) _c (x, y)));

[0083] S _y (x, y) = Σ(m _c (x, y)*sin(θ) _c (x, y)));

[0084] θ(x, y) = atan2(S) _y (x, y), S _x (x, y));

[0085] Among them, S _x (x, y) are the weighted vectors and their projection components in the horizontal direction, S _y (x, y) are the weighted vector sum's projection components in the vertical direction, m _c (x, y) represents the feature gradient magnitude of the c-th channel at coordinate position (x, y), θ _c (x, y) is the feature gradient direction angle of the c-th channel at coordinate position (x, y), Σ represents the summation operation over all channels, atan2 is the bivariate arctangent mapping function, and θ(x, y) is the principal gradient direction angle of the corresponding pixel position.

[0086] Step 304: Construct a local orientation histogram based on the principal gradient direction angle, and determine the pixels in the local orientation histogram that meet the preset peak conditions as contour intersection points.

[0087] Specifically, the principal gradient direction angles of all pixels within a circular neighborhood are used to construct the orientation histogram. The 360-degree orientation space is divided into 36 equal intervals, each spanning 10 degrees. Each neighboring pixel is weighted by its comprehensive feature gradient magnitude and its values ​​are accumulated in the corresponding interval to form a 36-dimensional local orientation histogram.

[0088] Peak detection is performed after applying a circular Gaussian smoothing with a window width of 3 intervals to the histogram. Intervals with amplitudes exceeding 60% of the global maximum value of the histogram and higher than the values ​​of their adjacent intervals are marked as valid peaks. A preset peak condition is met when there are at least 3 valid peaks and the angular distance between any two adjacent peaks is not less than 45 degrees. For pixels that meet this condition, non-maximum suppression is performed using a circular window with a radius of 5 pixels, and the pixel with the maximum total energy of the local histogram peaks is output as the contour intersection point.

[0089] In one possible implementation, after obtaining the contour intersection point, the method further includes:

[0090] Step 305: Extract branch segments from the contour intersection points along the contour skeleton line set, and combine them with human skeletal information to determine the limb axes corresponding to the connections of each limb.

[0091] For each contour intersection point, based on the principal direction angle of each branch, the corresponding skeleton line is traced, and a pixel sequence extending 20 pixels outward from the intersection point is extracted as a branch segment. Simultaneously, using the coordinate lines of each physical joint in the human skeleton information obtained from the preprocessor network, 16 limb axes corresponding to the bones are constructed.

[0092] As an improvement to the above scheme, when processing truncated edges or extremely short branches, if the length of the skeleton line obtained through tracking is less than 20 pixels, a dynamic sampling alternative scheme is activated. The actual available pixel length is recorded as L, and the number of sampling points is adaptively reduced to the maximum value between half of L rounded down and the constant 3, ensuring that at least 3 sampling points are retained to maintain the statistical significance of feature aggregation.

[0093] Step 306: Extract the mean values ​​of local features of the spatial positions corresponding to the branch segments and limb axes from the multi-scale feature map set, and use them as the orientation feature signature and skeletal feature signature, respectively.

[0094] Performing region feature aggregation on low-resolution high-dimensional feature maps can transform purely geometric pixel lines into semantic vectors with advanced anatomical properties, providing a benchmark for feature space comparison in solving topological hybridity problems.

[0095] Specifically, the process includes the following steps:

[0096] Step 3061: Perform equally spaced sampling along the extension direction of the branch segment or limb axis to obtain multiple sampling location points.

[0097] Within the extracted length range, the physical curve or straight line is divided into equal segments, and 10 spatial coordinate points are extracted on the branch segment and the limb axis as sampling location points.

[0098] Step 3062: In the specified resolution level of the multi-scale feature map set, extract the local feature vectors corresponding to each sampling location point using bilinear interpolation.

[0099] The specified resolution level is preferably a high-level semantic layer feature tensor with an input size of 1 / 8 and 512 channels. Since the sampled spatial coordinates are usually not integers, a bilinear interpolation algorithm is used to calculate the 512-dimensional data structure, i.e., the local feature vector, at the corresponding sub-pixel coordinates by combining the feature values ​​of the four nearest neighbor pixels.

[0100] Step 3063: Perform element-wise average pooling on each local feature vector belonging to the same branch segment or the same limb axis, compressing them into single vectors, which are used as directional feature signatures and skeletal feature signatures, respectively.

[0101] For the ten 512-dimensional local feature vectors obtained, the arithmetic mean of the corresponding elements is calculated along the channel dimension. The resulting 512-dimensional single vector has global semantic smoothness properties. The single vector corresponding to the contour branch is denoted as the directional feature signature, and the single vector corresponding to the limb skeleton is denoted as the skeletal feature signature.

[0102] Step 307: Calculate the semantic similarity between the directional feature signature and the skeletal feature signature in the feature space, and apply a distance decay penalty by combining the spatial vertical distance from the branch segment to the limb axis to obtain the attribution affinity.

[0103] When calculating the consistency index between different feature vectors, relying solely on semantic similarity may misidentify spatially distant but textured background contours as foreground limbs. Therefore, an exponential decay model based on spatial distance is introduced, with the specific formula for calculating attribution affinity expressed as:

[0104] A(b _i , l _j )=((f _bi *f _lj ) / (||f _bi ||*||f _lj ||))*exp(-d _perp 2 / (2*σ _d 2 ));

[0105] Among them, A(b) _i , l _j f represents belonging affinity. _bi Let f be the directional feature signature vector of the i-th branch. _lj Let f be the skeletal feature signature vector of the j-th limb, (f _bi *f _lj ) / (||f _bi ||*||f_lj The ||) term represents the quotient of the product of the dot product and the product of the magnitudes of two vectors, used to calculate the cosine similarity, d _perp σ is the average vertical distance from all sampling points on the branch segment to the corresponding limb axis. _d This is the distance attenuation scale parameter used to control the intensity of spatial penalty. The preset distance attenuation scale parameter can be set to 30 pixels.

[0106] Step 308: Match and assign branch segments based on affiliation affinity, and split the mixed skeleton lines at the contour intersection point to obtain the limb affiliation contour segment.

[0107] A two-dimensional affinity matrix is ​​constructed based on the number of branches and limbs. A greedy strategy is used to extract the global maximum value in the matrix to lock matching pairs. A filtering threshold of 0.3 is set, and branches below this threshold are judged as background and removed. After matching, according to the label of the limb, the original intersecting connection nodes are cut off at the physical coordinate level, and the independent limb-assigned contour segments are output.

[0108] In one possible implementation, the process of determining occlusion sequence information after obtaining the limb-assigned contour segment includes:

[0109] Step 309: At the intersection of contours, extract probability profile lines from the human body coarse contour probability map along a direction perpendicular to the axis of the assigned limb.

[0110] After clarifying the hierarchical relationships of each branch, the pixel line segment containing the intersection point and perpendicular to the axis of the limb being referred to in front is extracted as the sampling path. The total length of the probability profile line is set to 40 pixels, extending 20 pixels to each side from the contour intersection point, and then the probability values ​​of each pixel on this path are extracted.

[0111] Step 310: The proportion of pixels with probability values ​​not lower than a first preset threshold on the probability profile line is used as the contour integrity, and the existence of multiple consecutive intervals with probability values ​​lower than a second preset threshold is detected as contour interruption markers. Here, "multiple" refers to at least two intervals, but in some embodiments, it can be three, and in some embodiments, it can be four.

[0112] The first preset threshold is set to 0.5, and the second preset threshold is set to 0.2. The probability profile is traversed, and the ratio of the number of pixels with a probability value greater than or equal to 0.5 to the total number of pixels is calculated, which is recorded as the contour integrity. Simultaneously, sliding detection is performed in the sequence; when three or more consecutive pixels are found to have a probability value less than 0.2, a contour interruption marker representing a cliff signal is generated.

[0113] Step 311: Based on the priority comparison between the contour integrity of each limb at the contour intersection point and the contour interruption marker, determine the occlusion sequence information before and after each limb.

[0114] Specifically, the priority comparison follows the stacking determination rules.

[0115] First-level comparison interruption status: If the first limb does not show an outline interruption marker while the second limb does, it is determined that the first limb is located in front of the occlusion.

[0116] The second level of comparison completeness: If the two interruption states are the same, the limb with the larger outline completeness value is determined to be in front of the occlusion.

[0117] The third-level comparison probability mean: If the difference in contour integrity values ​​does not exceed 0.1, the arithmetic mean of the probability pixels on the probability profile line is calculated. The limb with the larger mean value is located in front of the occlusion. Through multi-level comparison, the occlusion sequence information with integer encoding is output. The larger the value, the closer it is to the observation device.

[0118] Example 4 describes how to solve the problem of corner sharpness loss caused by convolution operations by constructing an anisotropic Gaussian kernel, and the specific process of performing sub-pixel geometric correction after feature re-aggregation.

[0119] In one possible implementation, the process of locating the concave angle region in the step of feature re-aggregation on the concave angle region of the limb's associated contour segment is as follows: Figure 4 As shown, it includes:

[0120] Step 401: Retrieve the joint positions corresponding to the connections of each limb based on the human skeletal information, and calculate the discrete curvature in the corresponding neighborhood interval of the contour segment to which the limb belongs.

[0121] After obtaining contour data with a clear physical affiliation, the system traverses each contour segment and uses human skeletal information containing topological structure to locate the two-dimensional coordinates of the physical joints. For example, for the elbow joint connecting the upper arm and forearm, contour segment intervals within a 20-pixel range on both sides of the joint are extracted. For each pixel in the coordinate sequence of this contour segment, an arc is fitted with the five pixels before and after it, and the reciprocal of the radius of the fitted arc is calculated as the discrete curvature estimate at that point.

[0122] Step 402: Locate the concave corner vertex within the concave corner region based on the concave-convex direction and curvature extremum corresponding to the sign of the discrete curvature.

[0123] When determining the sign of discrete curvature, the coordinate sequence of contour segments is traversed in a counter-clockwise direction along the outer contour of the human body. Based on the right-hand rule, for the current pixel, two direction vectors are extracted: one pointing from its predecessor to the current point, and the other pointing from the current point to its successor. The cross product of these two vectors is then calculated. If the cross product is negative, it indicates that the contour bends to the right at that point, corresponding to a concave direction towards the inner side of the joint bend. Within this neighborhood, pixels with negative signs and absolute curvature values ​​higher than a preset threshold are selected. The preset threshold can be set to three times the overall average curvature of the contour segment. The continuous interval covered by high-curvature concave pixels is marked as a concave bend region, and the pixel with the largest absolute curvature value within this region is identified as the concave bend vertex.

[0124] Step 403: Determine the angle bisector direction based on the local tangent directions of the contours on both sides of the concave corner vertex, and divide the neighborhood of the concave corner vertex into the positive half-plane and the negative half-plane using the angle bisector direction as the boundary.

[0125] Using the concave vertex as the origin, extract contour segments of 10 pixels in length to each side of the vertex within its coverage area. Perform least-squares line fitting independently on the two segments to obtain two local tangent directions. Summate the unit vectors of the two tangent directions; the direction angle of the resulting vector is the angle bisector direction. Using the line passing through the vertex and having this direction angle as the physical dividing line, divide the circular region centered on the vertex with a radius of 12 pixels into two independent regions of equal area, denoted as the positive half-plane and the negative half-plane, respectively.

[0126] Step 404: In the positive half-plane and the negative half-plane, construct Gaussian kernels that are compressed along the local tangent direction and broadened along the contour normal direction, respectively, to obtain positive and negative anisotropic aggregated kernel pairs.

[0127] The isotropic receptive field of standard convolution simultaneously covers features from different surface orientations on both sides of a concave corner, resulting in the smoothing and blunting of sharp gradients. This step constructs an elliptical Gaussian kernel within the positive and side half-planes:

[0128] K _+ (dx, dy) = exp(-(dx*cos(θ)) _+ )+dy*sin(θ _+ )) 2 / (2*σ _t 2 )-(-dx*sin(θ _+ )+dy*cos(θ _+ )) 2 / (2*σ _n 2 ));

[0129] Among them, K _+(dx, dy) is the positive-lateral anisotropic Gaussian aggregation kernel, dx and dy are the horizontal and vertical offsets of the neighboring pixels relative to the concave corner vertex, exp is an exponential function with the natural constant as the base, and θ _+ σ is the local tangent direction angle of the front and side profiles. _t Let σ be the Gaussian standard deviation along the tangent direction. _n is the Gaussian standard deviation along the normal direction.

[0130] To ensure a narrow convergence range in the tangential direction and a wide range in the normal direction, σ is set. _t For 2 pixels, σ _n It is 6 pixels. A symmetric parameterized construction operation is performed on the negative half-plane to generate a negative anisotropic Gaussian aggregation kernel, which is then combined with the positive half-plane to form a positive and negative anisotropic aggregation kernel pair.

[0131] Among them, the feature re-aggregation adopts an anisotropic feature re-aggregation method, such as Figure 5 As shown, it includes the following steps 1-3:

[0132] Step 1: Using positive and negative anisotropic aggregation verification, with the concave corner vertex as the center, perform weighted average aggregation on the features in the multi-scale feature map set in the positive and negative half-planes respectively to obtain positive aggregated features and negative aggregated features.

[0133] The coordinates of the concave corner vertices are scaled and mapped onto a feature tensor with a spatial resolution of 1 / 4 of the original input, extracting 256-dimensional feature vectors for all locations within a circular region with a radius of 12 pixels. Within the pixel set covered by the positive half-plane, each feature vector is multiplied and added using the corresponding kernel function weights to obtain the positive side aggregated feature reflecting the gradient distribution along the normal direction on the surface of that side of the limb. Similarly, negative side aggregated features are generated within the negative half-plane.

[0134] Step 2 involves inputting the positive and negative aggregated features into a gating network that includes an attention gating mechanism, and outputting the dynamic fusion weights of the positive and negative half-planes. This process includes the following steps:

[0135] The positive-side aggregated features and the negative-side aggregated features are concatenated along the channel dimension to obtain the concatenated feature vector.

[0136] The extracted 256-dimensional positive side feature vector and 256-dimensional negative side feature vector are directly concatenated along the channel depth axis to generate a concatenated feature vector with a dimension of 512.

[0137] Furthermore, the concatenated feature vector is input into a gated network composed of fully connected layers for feature mapping and activation, and two scalar parameters that satisfy the summation and normalization conditions are output.

[0138] The gated network consists of two cascaded fully connected layers. The first layer maps the 512-dimensional input to 128 dimensions and applies a linear rectified activation function. The second layer maps the 128-dimensional input to 2 dimensions and applies a normalized exponential function to ensure that the sum of the two output values ​​is always equal to 1.

[0139] Based on this, two scalar parameters are used as dynamic fusion weights for the positive and negative half-planes, respectively.

[0140] The weighting is designed to take into account that when the feature confidence on the concave side decreases due to image noise or external occlusion, the gating mechanism can automatically reduce the value of the corresponding scalar parameter on that side, thus avoiding interference from inferior signals with the final fusion result.

[0141] Step 3: Perform cross-side adaptive fusion of positive and negative aggregated features based on dynamic fusion weights to obtain concave angle fusion features.

[0142] The cross-side adaptive fusion calculation formula is as follows:

[0143] F(v)=w _+ *F _+ (v)+w _- *F _- (v);

[0144] Where F(v) is the concave angle fusion feature, w _+ F represents the dynamic fusion weights corresponding to the positive and negative half-planes. _+ (v) represents the positive-side aggregation feature, w _- F represents the dynamic fusion weights corresponding to the negative half-plane. _- (v) represents the negative-side aggregation feature.

[0145] In one possible implementation, the step of obtaining the concave corner contour segment further includes correcting the coordinates of the concave corner vertex, including the following process:

[0146] By inputting the concave corner fusion features into a multilayer perceptron network, the sub-pixel offset from the concave corner vertex to the true precise location is predicted.

[0147] The multilayer perceptron network comprises three fully connected layers. The input layer receives 256-dimensional concave corner fusion features, which are then passed through the first hidden layer to output 128-dimensional features with linear rectified activation, and through the second hidden layer to output 64-dimensional features with linear rectified activation. The output layer produces 2-dimensional horizontal and vertical offsets, and without imposing non-linear activation function constraints, allowing the predicted sub-pixel offsets to cover the full value space, including both positive and negative values.

[0148] Furthermore, the sub-pixel offset is superimposed on the coordinates of the concave corner vertex to obtain the corrected concave corner vertex.

[0149] In a two-dimensional Cartesian coordinate system, the original extracted integer pixel coordinates are simply added to the predicted two-dimensional floating-point offsets to output the corrected concave vertex coordinates with sub-pixel precision.

[0150] Based on this, using the corrected concave corner vertex as the anchor point, the coordinate sequence of the limb belonging to the contour segment in the concave corner region is smoothed and resampled, so that the contour line converges to the position of the corrected concave corner vertex, thus obtaining the concave corner restored contour segment.

[0151] Directly replacing vertex coordinates can lead to discontinuous inflection points in the local geometry. Therefore, the coordinates of contour points within a 5-pixel range on both sides of the vertex are extracted, and a cubic spline interpolation algorithm is used to regenerate the distribution curve by using the corrected concave corner vertex as the center anchor point for forced passage. The newly generated smooth sequence is then used to replace the original coordinate sequence, and the final output sharpness yields a naturally restored concave corner contour segment.

[0152] Example 5 describes how, when a contour breaks due to external occlusion, the geometric parameters of the break endpoints are accurately inferred and corrected at the sub-pixel level by extracting multi-source geometric features, fitting a curvature evolution model, and fusing prior morphological distributions.

[0153] In one possible implementation, the step of inferring the geometric information of the fracture endpoint caused by occlusion, including the process of extracting the curvature evolution trend of the fracture endpoint within the visible region, comprises:

[0154] Step 501: Recover the contour segment from the concave corner and backtrack to the visible interval and sample at equal intervals to obtain the discrete curvature of each sampling point.

[0155] Specifically, the location of the fracture endpoint is based on the topological analysis results of the preceding steps. In the contour interruption marker detection of the aforementioned embodiment, it has been determined that there are interval positions on the contour segment to which each limb belongs where the probability drops sharply due to occlusion. The terminating pixel of the contour segment coordinate sequence corresponding to this interval position is the fracture endpoint of the limb contour. After concave corner recovery processing, the coordinate position of the fracture endpoint is inherited from the first or last pixel of the corresponding coordinate sequence in the concave corner recovered contour segment.

[0156] After identifying the break point caused by background object occlusion, the contour segment is recovered along the concave angle and traced backward towards the unoccluded visible area, starting from this break point. Within this backward tracing trajectory, multiple physical coordinate points are extracted at fixed pixel intervals. For each acquired sampling point, a local arc curve is fitted using the coordinates of its preceding and following pixels, and the reciprocal of the calculated arc radius is used as the discrete curvature at that location. This value reflects the true geometric curvature of the local contour on the two-dimensional plane.

[0157] Step 502: Extract the component magnitude change rate of the feature gradient at the corresponding location along the contour normal direction from the multi-scale feature map set, as the original feature curvature, specifically including:

[0158] Step 5021: Use the edge detection convolution operator to obtain the horizontal and vertical feature gradients on the multi-scale feature map set, and determine the normal direction based on the local tangent direction of each sampling point in the contour segment.

[0159] Extracting high-dimensional gradients in the feature space can compensate for the oversensitivity of single geometric curvature to discrete noise. The system extracts a feature tensor with a spatial resolution of one-quarter of the original input image size. Edge detection operators are applied to the data matrices of each channel within this tensor, and the rates of change of values ​​on the horizontal and vertical axes are calculated to obtain the horizontal and vertical feature gradients. Simultaneously, the local tangent direction angle is calculated based on the coordinate difference between adjacent points in the two-dimensional image coordinate system, and this local tangent direction angle is converted to an orthogonal normal direction angle by adding 90 degrees.

[0160] Step 5022: Calculate the projection components of the horizontal and vertical feature gradients along the normal direction, and combine the projection components of each channel to obtain the comprehensive normal gradient magnitude.

[0161] Extract the component of the feature gradient along the geometric normal and filter out redundant feature responses that are parallel to the contour direction.

[0162] G _nc =G _xc *cos(θ _n )+G _yc *sin(θ _n );

[0163] Among them, G _nc G is the projection component of the characteristic gradient along the normal direction. _xc For the horizontal feature gradient, G _yc For the vertical feature gradient, θ _n It represents the direction angle of the normal.

[0164] After obtaining the projection component of a single channel, the data from all feature channels are comprehensively calculated.

[0165] g _n =sqrt(Σ(G _nc 2 ));

[0166] Among them, g _n To represent the magnitude of the combined normal gradient, Σ indicates the summation over all channels of the feature map set, and G... _nc is the projection component of the characteristic gradient of a single channel along the normal direction, and sqrt represents the square root operation.

[0167] Step 5023: Using the arc length interval between adjacent sampling points as the step size, calculate the first-order finite difference quotient of the comprehensive normal gradient magnitude along the contour arc length direction, and use the first-order finite difference quotient as the original feature curvature.

[0168] In flat contour regions, the magnitude of the composite normal gradient is approximately constant along the arc length, while in high curvature regions, rapid rotation in the normal direction causes this magnitude to change drastically. Therefore, its rate of change can be used as a substitute indicator for measuring curvature.

[0169] k _feat_raw =(g _n_next -g _n_prev ) / (2*ds);

[0170] Where, k _feat_raw For the original characteristic curvature, g _n_next For the combined normal gradient magnitude of subsequent sampling points, g _n_prev ds represents the combined normal gradient magnitude of the preceding sampling point, and ds represents the arc length interval between adjacent sampling points.

[0171] Step 503: Using a pre-established linear calibration model, the original feature curvature is mapped to the same dimension as the discrete curvature to obtain the calibration feature curvature.

[0172] Since the unit of the original feature curvature is calculated based on the feature response intensity, it does not match the physical reciprocal unit of the discrete curvature in terms of dimensions and cannot be directly fused. By statistically analyzing the distribution relationship between the true geometric curvature and the original feature curvature in a large number of training samples, a least-squares linear regression equation is constructed for numerical calibration.

[0173] k _feat =α*k _feat_raw +β;

[0174] Where, k _feat To calibrate the characteristic curvature, α is the linear calibration scaling factor, and k _feat_raw β represents the original characteristic curvature, and β is the linear calibration bias coefficient.

[0175] Step 504: Weighted fusion of discrete curvature and calibrated characteristic curvature is performed to obtain robust curvature estimates for each sampling point.

[0176] Perform arithmetic addition with fixed weights on the two sets of curvature data after unifying dimensions.

[0177] k _robust =0.7*k _geo +0.3*k _feat ;

[0178] Where, k _robust k is the robust curvature estimate._geo For discrete curvature, k _feat To calibrate the feature curvature, this fusion mechanism injects noise-resistant and smoothing capabilities into high-dimensional features while maintaining geometric accuracy.

[0179] Step 505: Fit a polynomial evolution model using the robust curvature estimate and the sampled arc length, and extrapolate the initial estimate of the tangent direction and the initial estimate of the curvature at the fracture endpoint based on the polynomial evolution model.

[0180] The obtained robust curvature estimates and their corresponding sampled arc length coordinates are used to form a data pair set. The least squares method is used to optimize the error and fit an evolution equation describing the curvature as a function of distance.

[0181] k(s)=a _0 +a _1 *s+a _2 *s 2 ;

[0182] Where k(s) is the curvature value at arc length coordinate s, a _0 a is the coefficient of the constant term. _1 a is a first-order coefficient. _2 is a second-order coefficient, and s is the arc length coordinate measured with the fracture endpoint as the origin.

[0183] After parameter fitting, the coordinate variables are set to zero and substituted into the polynomial model to obtain the initial estimate of curvature at the current endpoint. Simultaneously, utilizing the differential relationship between curvature and tangent direction angle, analytical integration is performed on the polynomial evolution model along the arc length direction, starting from the sampling point farthest from the fracture endpoint within the visible interval. The integral result is used to correct the known tangent direction angle at that sampling point, thus obtaining the initial estimate of the tangent direction at the fracture endpoint. This yields the initial estimates of curvature and tangent direction at the current endpoint.

[0184] Optionally, by combining the geometric constraints of the modified concave vertex with the pre-statistical prior distribution of limb morphology, the tangent direction and curvature at the fracture endpoint are inferred to obtain the endpoint geometric constraint parameters, specifically including:

[0185] (1) On the concave corner recovery contour segment, retrieve the corrected concave corner vertex that is closest to the fracture endpoint and use it as the curvature anchor point;

[0186] Simple polynomial extrapolation results in larger errors as the distance increases, necessitating the search for a deterministic geometric structure as a constraint benchmark. By searching along the same contour segment path, the joint vertices whose sharpness recovery was completed in the previous steps are extracted, and their precise curvature values ​​and arc length coordinates from the endpoints are recorded.

[0187] The precise curvature value of the concave corner vertex is obtained by performing local circular arc fitting calculation on the coordinate sequence of the vertex's neighborhood on the recovered contour segment of the concave corner. The calculation method is the same as the method for obtaining discrete curvature in the aforementioned embodiment. Since the vertex coordinates have undergone sub-pixel correction and smooth resampling, their discrete curvature accuracy is higher than that of ordinary contour pixels, making them suitable as a high-confidence curvature anchoring reference.

[0188] (2) Introduce an exponential decay correction term centered on the curvature anchor point into the polynomial evolution model, and require that the value of the calibrated model at the curvature anchor point be equal to the precise curvature of the anchor point, and perform equality constraint calibration.

[0189] To ensure that the fitted curve passes through this deterministic geometric vertex without disrupting the trend at the far end, a distance-based negative exponential decay function is introduced.

[0190] k'(s)=a _0 +a _1 *s+a _2 *s 2 +λ*exp(-|ss _a | / 15);

[0191] Where k'(s) is the curvature value of the calibrated model at arc length s, and a _0 a _1 With a _2 λ is the coefficient of the original polynomial, λ is the calibration correction intensity coefficient, exp is the exponential function with the natural constant as the base, and s _a is the arc length coordinate of the curvature anchor point, and 15 is the attenuation scale parameter fixed at 15 pixels.

[0192] By substituting the precise curvature of the anchor point into the equation, a uniquely determined calibration correction strength coefficient can be obtained.

[0193] (3) Based on the calibrated model and the fitting residual, extract the calibrated curvature estimate and the corresponding observation uncertainty at the fracture endpoint.

[0194] The coordinate variables are set to zero and substituted into the calibrated model to output the updated curvature prediction values ​​at the endpoints. Simultaneously, the variance distribution between the fitted sample points and the evolutionary model is calculated, and the standard deviation corresponding to this variance is used as the observation uncertainty to quantify the quality of the extrapolation.

[0195] As an alternative to the above scheme, when the concave corner recovery profile segment fails to find the corrected concave corner vertex, the system will skip the equality constraint calibration step that introduces the exponential decay correction term, directly extract the constant term coefficient obtained by extrapolation of the original polynomial as the observation benchmark, and will not perform subsequent expansion adjustment on the original observation uncertainty.

[0196] In one possible implementation, after extracting observation uncertainties, the method further includes:

[0197] (4) Based on the arc length distance from the curvature anchor point to the fracture endpoint, the observation uncertainty is adjusted by an expansion that monotonically increases with the arc length distance to obtain the expansion observation uncertainty.

[0198] The farther the curvature anchor point is from the endpoint, the weaker the physical effectiveness of the calibration mechanism becomes, and the random uncertainty of the model must be amplified to prevent false accuracy.

[0199] σ _obs_prime =σ _obs *exp(|s _a | / 30);

[0200] Where, σ _obs_prime For the uncertainty of expansion observation, σ _obs The original observational uncertainty is represented by exp, which is an exponential function with the natural constant as its base. _a is the arc length coordinate of the curvature anchor point, and 30 is the distance attenuation scale parameter that controls the rate of error expansion.

[0201] (5) The calibration curvature estimate is used as the expectation of the observation distribution. Combined with the uncertainty of the dilated observation, it is fused with the prior distribution of limb morphology by Bayesian posterior fusion to obtain the final inferred curvature value.

[0202] The current observation data is integrated with the Gaussian prior distribution based on a large number of normal human anatomical morphologies within a posterior probability framework.

[0203] k _final =(k _prime_0 / σ _obs_prime 2 +μ _t / σ _p 2 ) / (1 / σ _obs_prime 2 +1 / σ _p 2 );

[0204] Where, k _final For the final inferred curvature value, k _prime_0 σ is the estimated endpoint curvature after calibration. _obs_prime To account for the uncertainty in the expansion observation, μ _t Let σ be the prior mean of the curve of the prior distribution of limb morphology. _p Let be the a prior standard deviation of the curve of the prior distribution of limb morphology.

[0205] To facilitate understanding of the Bayesian posterior curvature inference process, a simplified normalized example is provided. Assume that after fitting a quadratic polynomial to a fracture endpoint and performing exponential decay calibration, the estimated calibrated curvature at the endpoint is 0.6, and the original observation uncertainty is 0.1. The absolute value of the arc length distance of the curvature anchor point is 15 pixels. Substituting this into the uncertainty dilation formula, the dilated observation uncertainty is calculated to be approximately 0.165. The pre-statistical a priori mean of curvature for this limb category is 0.4, and the a priori standard deviation of curvature is 0.1. Substituting these values ​​into the Bayesian posterior fusion formula, the final inferred curvature value is approximately 0.454. This result objectively demonstrates the physical characteristic that when the observation uncertainty is higher than the a priori standard deviation, the inference result converges towards the anatomical statistical prior.

[0206] (6) The inferred curvature value and the inferred tangent direction angle inferred based on the initial estimate of the tangent direction are used together to form the endpoint geometric constraint parameters.

[0207] Optionally, the initial estimate of the tangent direction is corrected for curvature consistency based on the inferred curvature value to obtain the inferred tangent direction angle; the inferred curvature value and the inferred tangent direction angle together constitute the endpoint geometric constraint parameters.

[0208] For the inference of the tangent direction, the initial estimate of the tangent direction is used as a benchmark, and the curvature change of the arc segment in the neighborhood of the endpoint is integrally corrected by combining the inferred curvature value to obtain the inferred tangent direction angle that matches the final inferred curvature value.

[0209] Specifically, using the fundamental theorem of differential geometry that considers curvature as the first derivative of the tangent direction angle with respect to arc length, the inferred curvature value on the arc segment near the endpoint is numerically integrated along the arc length direction. The integration result is then superimposed onto the initial estimate of the tangent direction, thus obtaining the inferred tangent direction angle that geometrically matches the inferred curvature value. It should be understood that this correction process utilizes the fundamental differential geometric relationship between curvature and the tangent direction angle.

[0210] The numerical values ​​obtained from statistical and geometric calibrations are packaged and output as a structured parameter descriptor that provides rigorous physical boundary conditions for subsequent curve stitching operations.

[0211] Optionally, before stitching the concave corner recovered contour segments according to the endpoint geometric constraint parameters and occlusion sequence information, an endpoint sub-pixel correction step is also included:

[0212] The original tangent direction at the fracture endpoint is replaced with the inferred tangent direction angle in the endpoint geometric constraint parameters. Using this inferred tangent direction angle as the direction boundary condition, smooth resampling is performed on the local coordinate sequence of the fracture endpoint to obtain the direction-corrected local coordinate sequence of the endpoint.

[0213] After obtaining the target tangent direction angle, it is used as a first-order derivative constraint condition and input into the cubic spline interpolation function to refit and sample the pixel coordinate set of the endpoint and its neighboring region, ensuring that the geometric line achieves a smooth numerical transition to the target direction and eliminating the angle caused by abrupt changes in direction.

[0214] Furthermore, the difference between the inferred curvature value in the endpoint geometric constraint parameters and the original discrete curvature of the fracture endpoint is calculated, and this difference is converted into the corresponding curvature radius difference.

[0215] Alternatively, the discrete curvature of the fracture endpoint on the concave corner recovery profile segment is extracted as the original curvature of the endpoint. The difference between the inferred curvature value in the endpoint geometric constraint parameters and the original curvature of the endpoint is calculated, and the difference is converted into the corresponding curvature radius difference.

[0216] Curvature is the reciprocal of radius. By calculating the difference between the inferred curvature value and the original observed curvature value, the absolute deviation of the geometric arc on the physical radius scale can be obtained.

[0217] Based on this, a subpixel translation correction based on the difference in curvature radius is applied to the coordinates of the fracture endpoints along the contour normal direction.

[0218] Alternatively, the fracture endpoint coordinates in the local coordinate sequence of the oriented corrected endpoints are adjusted by applying a subpixel translation correction based on the difference in the radius of curvature along the contour normal direction to obtain the corrected fracture endpoint coordinates.

[0219] After obtaining the absolute deviation of the physical radius, the two-dimensional coordinates of the endpoint are translated linearly at the sub-pixel level along the normal vector direction orthogonal to the endpoint profile, according to the deviation value. When the curvature is corrected to increase, the translation is made towards the inside of the physical bend; when it decreases, the translation is made towards the outside, ultimately completing the high-precision geometric position restoration of the fracture endpoint in the two-dimensional image space.

[0220] Example 6 further describes the specific process of splicing the scattered contour segments according to curvature continuity after completing local feature extraction and endpoint geometric inference, and performing global graphics rendering according to the spatial occlusion sequence to finally generate a human body external contour map with high fidelity.

[0221] In one possible implementation, the concave corner contour segments are stitched together based on endpoint geometric constraint parameters and occlusion sequence information to output a human body contour boundary map, including:

[0222] Step 601: Based on the inferred tangent direction angle in the endpoint geometric constraint parameters, a transition connection curve for curvature continuity splicing is generated between concave angle recovery contour segments belonging to the same limb but truncated by the occluded gap, forming a complete limb contour line.

[0223] Specifically, the same physical limb may be divided into several spatially disconnected contour coordinate sequences due to one or more occlusion zones. The system groups the data based on the limb category identifier carried by each contour segment. For two contour segments that belong to the same limb and are adjacent on the spatial physical axis after grouping, that is, the tail end of the previous contour segment and the head end of the next contour segment are arranged sequentially in the limb direction, and a transition connection curve is constructed in their occlusion gap area.

[0224] In this embodiment, the transition connection curve is specifically constructed using a cubic Bézier curve. The coordinates of the tail end of the preceding contour segment are set as the starting point of the curve, and the coordinates of the head end of the following contour segment are set as the ending point of the curve. The inferred tangent direction angles corresponding to the endpoint geometric constraint parameters are extracted and assigned to the control handle directions of the starting point and the ending point, respectively. Simultaneously, the Euclidean straight-line distance between the starting and ending points is calculated, and 1 / 3 of this straight-line distance is set as the physical length constraint of the control handle.

[0225] This calculation process takes into account that using Bézier curves that include the inferred tangent direction from the endpoints for bridging naturally ensures, mathematically, that the curve maintains first-order continuity with the tangent directions of the known contour segments at both ends. Furthermore, the parameter configuration of the 1 / 3 length constraint makes the curvature change of the curve relatively gentle in the middle region of the occlusion gap, resulting in an overall visual effect approaching second-order geometric continuity. By sequentially concatenating each discrete contour segment with the transition curve between them on a coordinate sequence, a closed or semi-closed complete limb contour line can be output.

[0226] In some alternative implementations, for limb postures with more dramatic curvature changes, the transition connection curve construction method can be replaced by a B-spline curve fitting algorithm, which dynamically adjusts the local curvature weights by increasing the number of internal control points to adapt to filling more complex nonlinear geometric gaps.

[0227] Step 602: Based on the occlusion layer information, the complete outline of each limb is drawn by covering it in order of occlusion level from low to high, so as to obtain the layer arrangement outline map.

[0228] Occlusion sequence information is stored as integer values, with larger integer values ​​indicating that the physical limb is closer to the image viewing device in depth space. After acquiring 16 complete limb outlines containing all anatomical parts, the system initializes a uniform-sized two-dimensional blank canvas in memory.

[0229] Based on a monotonically increasing integer value order, the pixel coordinates of each contour line are sequentially mapped and drawn onto the 2D canvas. In this drawing loop, due to the rendering rule from low to high levels, when multiple limb contour lines spatially overlap in the same 2D pixel coordinate area, the pixel values ​​of the later-drawn high-level limb contours will directly cover and override the pixel values ​​of the earlier-drawn low-level limb contours.

[0230] Through the deep overlay rendering mechanism, the system retains only the physical boundaries of the visible limbs in front, so that the hidden parts of the occluded limbs in the back are naturally eliminated in the overlapping area, and finally the pixel matrix of the two-dimensional canvas is output as a layered outline map.

[0231] Step 603: Perform weighted average smoothing on the transition regions of connecting joints between adjacent limb contour lines in the hierarchical contour map, and output the human body contour boundary map.

[0232] After completing the hierarchical coverage, global consistency needs to be maintained at the joints of each independent limb component. Taking physical junctions such as the shoulder connecting the upper arm and torso, and the hip connecting the thigh and torso as examples, the system extracts the tangent direction angles of two adjacent limb contour lines at the corresponding joint points and calculates the absolute value of the angle difference between the two.

[0233] In this embodiment, the preset difference alarm threshold is set to 15 degrees. When the calculated angle difference is greater than 15 degrees, it is determined that there is an uneven geometric angle at the junction. At this time, a circular neighborhood with a radius of 10 pixels is defined with the coordinates of the joint point as the geometric center. All contour coordinate sequences within this neighborhood are extracted, and a Gaussian function is applied to calculate the weights. A weighted average smoothing operation of the spatial coordinates is then performed to eliminate the abrupt sharp angles at the junction.

[0234] After smoothing calculations are completed for all joint transition regions, the processed overall geometric curve is converted into a binary image matrix with a single-pixel width, consistent with the spatial resolution of the original scene image. In this binary image matrix, white pixels represent the determined physical contour boundary positions of the human body, and black pixels represent background areas without targets, resulting in a human body contour boundary map that balances topological correctness with geometric accuracy of joint concave angles.

[0235] Example 7 describes an offline preprocessing procedure, including network parameter optimization and anatomical prior statistics, performed before the formal human contour boundary extraction task to ensure the efficiency and high accuracy of the online inference process.

[0236] One possible implementation includes pre-training a feature extraction model and constructing a prior distribution before acquiring the original scene image, specifically divided into:

[0237] Phase 1: Using a training dataset containing contour and joint position annotations, perform multi-task end-to-end joint training on the feature extraction model.

[0238] Specifically, the entire offline training process is strictly divided into multiple time-series stages without circular dependencies. The training dataset consists of a massive number of complex scene images containing human figures, along with their corresponding pixel-by-pixel contour mask annotation data and two-dimensional physical coordinate annotation data for each joint.

[0239] The training dataset can be obtained from publicly available datasets, such as the COCO dataset (a large-scale object detection dataset containing various common objects and their contextual information) or the OCHuman dataset (a dataset focused on human instance detection and segmentation under severe occlusion), which contain public benchmark datasets with human keypoint annotations and instance segmentation annotations, or obtained through conventional data collection and annotation methods in this field. The specific size and distribution of the training data can be adjusted according to the actual application requirements.

[0240] In the first stage, the feature extraction model serves as a unified computation graph, performing end-to-end backpropagation and parameter updates.

[0241] The total training loss function in this stage is defined as the weighted sum of contour loss and skeleton loss.

[0242] L _total =λ _c *L _c +λ _s *L _s ;

[0243] Among them, L _total To train the total loss function, L _c For the contour loss of the contour decoding branch, L _s For the bone loss of the bone decoding branch, λ _c λ is the balancing weight for the contour loss. _s This sets the balancing weights for bone loss. In a specific system configuration, the balancing weight for contour loss can be set to 1.0, and the balancing weight for bone loss can be set to 0.5.

[0244] The contour loss consists of the sum of the binary cross-entropy loss and the local region overlap loss. The former is used to calculate the probability classification error per pixel, while the latter is used to measure the overall spatial consistency between the predicted contour and the ground truth contour, in order to alleviate the severe numerical imbalance between the distribution of contour pixels and background pixels.

[0245] The skeletal loss consists of the sum of the mean square error loss of the joint heatmap and the mean square error loss of the limb vector field. A preset optimizer is used to perform parameter iterations; specifically, the optimizer can be an adaptive moment estimation optimizer with an initial learning rate set to 0.0001, and the learning rate decays after a certain number of training epochs.

[0246] By introducing image enhancement strategies that include random scale scaling, horizontal flipping, and rotation, the generated multi-scale feature maps are endowed with feature representation capabilities that take into account both edge sensitivity and anatomical semantics.

[0247] After the first stage of training is completed, based on the curvature of the real occluded fracture endpoints in the training dataset, the prior distribution of limb morphology is generated by category statistics, and the network weights of the feature extraction model are fixed.

[0248] Meanwhile, the feature extraction model with fixed weights is used to perform forward inference on the training dataset to extract the data pairs of the original feature curvature and the true discrete curvature of the contour sampling points in each training sample. The proportional coefficient and bias coefficient of the linear calibration model are obtained by least squares linear regression fitting, and the linear calibration model is solidified as constant parameters in the inference stage.

[0249] After completing the first stage of joint optimization calculation, the update of the convolution kernel matrix and batch normalization parameters in the feature extraction model is stopped, the network weights are fixed, and the model is switched to a read-only state in subsequent optimization stages.

[0250] Simultaneously, the system performs offline statistical tasks for morphological parameters. It iterates through all image samples in the training dataset that contain annotations for external occlusion. These annotations can be obtained from public datasets containing instance-level occlusion relationship annotations (such as the occlusion instance segmentation annotations in the OCHuman dataset), or obtained by jointly annotating the occluded regions and complete contours of the training samples using conventional manual annotation methods. Based on the ground truth annotations of the complete contours and the location annotations of the occluded regions, the true curvature values ​​of the contours at each occlusion break point can be directly calculated.

[0251] For each predefined limb category, the true curvature value and true tangent direction angle value at the contour occlusion break point are extracted. The set of true curvature values ​​collected under the same category is fitted with a Gaussian probability density function to calculate the prior mean and standard deviation of curvature for that limb category. For all 16 limb categories, their respective Gaussian distribution feature parameters are generated sequentially and solidified into a constant lookup table, forming a pre-statistical prior distribution of limb morphology that can be directly called by the subsequent inference module.

[0252] This feature extraction operation compresses the statistical anatomy of large datasets into low-dimensional parameterized priors, avoiding repetitive statistical calculations in the online inference stage.

[0253] The second stage involves using a feature extraction model with fixed weights to perform forward inference on the training dataset to locate concave corner vertices. The actual coordinates of the concave corner vertices are used as supervision signals to drive independent spatial error loss. A gating network for feature re-aggregation and a multilayer perceptron network for sub-pixel relocalization are jointly trained.

[0254] In other words, the feature extraction model with fixed weights performs forward inference on the training dataset to locate the concave corner vertices in the curved concave corner region. The actual concave corner vertex coordinates are used as supervision signals to drive independent spatial error loss. The gating network for feature fusion in feature re-aggregation and the multilayer perceptron network for concave corner vertex coordinate correction are jointly trained.

[0255] To decouple the optimization objectives of global topological feature extraction and local geometric relocalization, the system sets up an independent second training phase. Using a feature extraction model with fixed parameters, the forward inference calculation process is re-executed on the training dataset, outputting the corresponding coarse contour probability map and skeletal feature vector. Based on the output data of the forward inference, the geometric bending and concave corner regions in the image are sequentially calculated and located, and high-dimensional neighborhood aggregation features of these regions are extracted.

[0256] The extracted features are input into the gating network and multilayer perceptron network to be initialized, predicting the corrected coordinates of the 2D concave corner vertices. The system extracts the accurate, unbiased joint corner coordinates from the data annotations as the supervision benchmark. An independent spatial error loss function is defined:

[0257] L _disp =(dx _pred -dx _true ) 2 +(dy _pred -dy _true ) 2 ;

[0258] Among them, L _disp For spatial error loss, dx _pred dy represents the horizontal subpixel offset predicted by the network. _pred dx represents the vertical subpixel offset predicted by the network. _true dy is the horizontal offset of the truth value. _true This is the vertical offset of the true value.

[0259] By backpropagating the gradient of the spatial error loss, the weights of the fully connected layers in the gated network and the mapping parameters in the multilayer perceptron network are updated independently. The optimizer can also be an adaptive moment estimation optimizer, with an initial learning rate set to 0.0005. Iterative calculations are performed on image samples containing concave corner regions based on independent batch sizes until the loss function converges numerically.

[0260] A rigorous temporal segmentation optimization strategy ensures that the feature extraction model can focus on the parsing of the global anatomical topology, and also guarantees that the relocation subnetwork responsible for micro-geometric repair is not subject to mathematical interference from global large-scale error gradients. When executing a new image extraction task online, the system directly loads all network weights and lookup table constants fixed in the previous stages and performs forward computation sequentially, with no circular dependencies in the computation graph between processing modules.

[0261] To facilitate understanding of the collaborative working process of the above embodiments, a typical scenario is given. Assume that in the original scene image, a pedestrian has their arms crossed in front of their chest, and a lamppost obscures the middle of the pedestrian's right thigh. After feature extraction, the coarse contour probability map shows multiple high-probability line segments in the area where the arms are crossed, and a sharp drop in probability values ​​in the lamppost-obscured area. Through topological analysis, the system uses feature signature affinity to assign the mixed line segments in the crossing area to the left and right arms respectively, and determines that the left arm is in front of the right arm based on the probability profile.

[0262] After concave angle restoration, the blunted contour corner points located on the inner sides of both elbows were restored to near-realistic physical sharpness by anisotropic kernel re-aggregation. Further endpoint inference was performed, and the two fracture endpoints at the point where the right thigh contour was truncated by the lamppost obtained Bayesian-calibrated tangent directions and curvature parameters. Finally, after stitching, the system smoothly bridged the lamppost-occluded area using Bézier curves and correctly rendered the thigh contour under the lamppost occlusion according to the layer sequence, outputting a complete human body external contour boundary map.

[0263] In typical complex occlusion testing scenarios, the human contour boundary map output by the method of this invention shows a significant improvement in topological accuracy compared to traditional pixel-by-pixel edge detection methods, particularly in the accuracy of attribution to limb intersection regions and the preservation of geometric sharpness in joint bending regions. Furthermore, through a Bayesian posterior constraint inference mechanism, contour continuation at occlusion breakpoints also outperforms pure numerical extrapolation methods in terms of curvature continuity and directional consistency. It should be understood that the specific performance improvement may vary depending on the complexity of the application scenario and the characteristics of the dataset.

[0264] This invention abandons branch tracing that relies on pure geometric features at the pixel level. Instead, it extracts orientation from a high-dimensional semantic space and performs affinity matching with skeletal feature signatures, then combines this with the distribution rules of probabilistic profile lines to quantify and determine the relationship before and after occlusion. This effectively isolates visually highly overlapping or parallel limb lines, precisely decomposing disordered, interconnected contours into independent line segments with clear anatomical hierarchies and spatial depth levels. It solves the problem of easily confused topological connectivity of human contours in complex occlusion scenarios.

[0265] To address the problem of easy loss of local geometric features, this invention achieves improvements in two dimensions.

[0266] Firstly, to address the blunting of joint concave angle sharpness caused by the isotropic receptive field of traditional feature extraction models, a half-plane anisotropic kernel segmented along the angle bisector is used for directional feature re-aggregation, supplemented by gated fusion and sub-pixel offset prediction. This mechanism physically blocks the cross-contamination of heterogeneous features on both sides of the concave angle, restoring the blurred pixel transition zone to a high-sharp physical corner point that approximates the real human body structure.

[0267] Secondly, in response to contour breaks and geometric divergences caused by external occlusion, this invention integrates robust curvature evolution from dual sources, anchor point calibration, and statistical morphological prior distribution to infer missing endpoint parameters within a Bayesian posterior framework. This eliminates the severe curling distortion caused by unidirectional numerical extrapolation at the distal end, allowing the geometric trend of the broken edges to naturally converge towards a statistical morphology that conforms to the laws of human anatomy, achieving a globally consistent reconstruction of the human body boundary with high fidelity and strong robustness.

[0268] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A method for extracting human contour boundaries in complex scenes based on deep learning, characterized in that, include: Obtain the original scene image; The original scene image is processed using a pre-trained feature extraction model to obtain a multi-scale feature map set, a coarse outline probability map of the human body, and human skeleton information; Topological analysis is performed based on multi-scale feature maps and human coarse contour probability maps to obtain the contour segments to which limbs belong and occlusion sequence information. Based on human skeletal information and multi-scale feature maps, feature re-aggregation is performed on the concave angle region on the contour segment to which the limb belongs to obtain the concave angle restored contour segment. Based on the geometric constraints on the contour segment recovered from the concave corner, the geometric information of the fracture endpoint caused by occlusion is inferred, and the endpoint geometric constraint parameters are obtained. Based on the endpoint geometric constraint parameters and occlusion sequence information, the concave corner contour segments are stitched together to output a human body contour boundary map.

2. The method according to claim 1, characterized in that, The original scene image is processed using a pre-trained feature extraction model to obtain a multi-scale feature map set, a coarse human outline probability map, and human skeletal information, including: By using a shared encoder to extract features from the original scene image at multiple levels, a multi-scale feature map set with progressively decreasing spatial resolution is obtained. The multi-scale feature map set is input into the contour decoding branch, and after step-by-step upsampling and feature concatenation, a coarse human contour probability map is generated. Input the multi-scale feature map set into the skeleton decoding branch to generate joint heat map sets and limb orientation vector fields; Joint coordinates are determined based on joint heatmaps, and limb direction vector fields are used to match adjacent joints to obtain human skeletal information.

3. The method according to claim 1, characterized in that, Topological analysis is performed based on multi-scale feature maps and human coarse contour probability maps to obtain the contour segments to which limbs belong and occlusion sequence information, including: Skeletalization tracing is performed on the coarse outline probability map of the human body to obtain a set of outline skeleton lines; Within the pixel neighborhood of the contour skeleton line set, the feature gradient direction and feature gradient magnitude of multiple channels are extracted from the multi-scale feature map set respectively. Using the feature gradient magnitude as the weight, calculate the weighted vector sum of the feature gradient directions of each channel on the unit circle, and determine the principal gradient direction angle of the corresponding pixel position accordingly. A local orientation histogram is constructed based on the principal gradient orientation angle, and pixels that meet the preset peak conditions are identified as contour intersection points.

4. The method according to claim 1, characterized in that, In the feature re-aggregation of the concave-angle region on the contour segment to which the limb belongs, the location of the concave-angle region includes: Based on the human skeletal information, the joint positions corresponding to the connections of each limb are retrieved, and the discrete curvature is calculated in the corresponding neighborhood interval of the contour segment to which the limb belongs. Based on the concave / convex direction corresponding to the sign of the discrete curvature and the curvature extremum, locate the concave corner vertex within the concave corner region; The direction of the angle bisector is determined based on the local tangent directions of the contours on both sides of the concave corner vertex, and the neighborhood of the concave corner vertex is divided into the positive half-plane and the negative half-plane using this as the boundary. In the positive and negative half-planes, Gaussian kernels are constructed that are compressed along the local tangent direction and broadened along the contour normal direction, respectively, to obtain positive and negative anisotropic aggregated kernel pairs.

5. The method according to claim 4, characterized in that, Feature re-aggregation employs anisotropic feature re-aggregation methods, including: Using positive and negative anisotropic aggregation verification, with the concave corner vertex as the center, weighted average aggregation is performed on the features in the multi-scale feature map set in the positive and negative half-planes respectively to obtain positive aggregated features and negative aggregated features. Input the positive and negative aggregated features into a gating network that includes an attention gating mechanism, and output the dynamic fusion weights of the positive and negative half-planes. Based on the dynamic fusion weights, the positive and negative aggregated features are adaptively fused across sides to obtain concave angle fused features.

6. The method according to claim 5, characterized in that, The steps to obtain the concave corner contour segment also include correcting the coordinates of the concave corner vertices: By inputting the concave corner fusion features into a multilayer perceptron network, the sub-pixel offset from the concave corner vertex to the true precise location is predicted. The subpixel offset is superimposed onto the coordinates of the concave corner vertex to obtain the corrected concave corner vertex; Using the corrected concave corner vertex as the anchor point, the coordinate sequence of the limb belonging to the contour segment within the concave corner region is smoothed and resampled, so that the contour line converges to the position of the corrected concave corner vertex, thus obtaining the concave corner restored contour segment.

7. The method according to claim 5, characterized in that, The positive and negative aggregated features are input into a gating network that includes an attention gating mechanism, and the output is a dynamic fusion weight between the positive and negative half-planes, including: The positive-side aggregated features and the negative-side aggregated features are concatenated along the channel dimension to obtain the concatenated feature vector. The concatenated feature vector is input into a gated network consisting of fully connected layers for feature mapping and activation, and outputs two scalar parameters that satisfy the summation and normalization conditions. Two scalar parameters are used as dynamic fusion weights for the positive and negative half-planes, respectively.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program. Wherein, when the executable program is running, it controls the device containing the computer-readable storage medium to perform the method described in any one of claims 1 to 7.

9. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 7.

Images