A background tampering detection method based on human posture and shadow kinematic constraint

Abstract

The application provides a background tampering detection method based on human posture and shadow kinematics constraint, comprising: acquiring a source image and a to-be-detected image, extracting human posture features, projected shadow features and global texture features; establishing a kinematic correlation between human posture and projected shadow according to a theoretical space migration trajectory and a human joint node displacement component; performing geometric transformation on the source image shadow to obtain predicted shadow features; comparing the predicted shadow features with actual shadows of the to-be-detected image to distinguish legal shadows from abnormal shadows and generate a mask; and performing mask area rejection on the global texture features, comparing the remaining pixel distribution at the same position, and determining a background tampering detection result. The method can solve the problem that existing detection cannot distinguish between reasonable shadow deformation and real background tampering, identify retained or fake shadows by establishing kinematic constraints between shadows and human postures, improve the accuracy and robustness of background tampering detection, and identify image authenticity from the perspective of shadow consistency.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for detecting background tampering based on human posture and shadow kinematic constraints. Background Technology

[0002] In the field of image authenticity verification based on human body segmentation and background reconstruction, background similarity detection is used to determine whether two images originate from the same real physical space, rather than being created through image cutout, pasting, or background replacement. Common applications include remote identity verification, video evidence authentication, and live stream review. Conventional detection methods first separate the human body from the image and then compare the similarity of the remaining background areas. However, the shadow cast by a human body on the ground or wall is neither the human body itself nor a true background texture, but rather a grayscale variation area formed when light is blocked by the human body. The shape of the shadow changes position, outline, and area with changes in human posture. This makes it impossible to distinguish between reasonable physical deformation of the shadow and actual background tampering when comparing the background similarity of two images. On the one hand, when the human posture changes, the position and shape of the shadow on the ground will inevitably change accordingly. At this time, the pixel brightness at the same coordinate position in the two images will produce a significant difference. Conventional detection mechanisms cannot distinguish whether this difference is a normal migration of the shadow or background replacement, thus judging genuine and legitimate images as having inconsistent backgrounds, resulting in false blocking. On the other hand, attackers might repeatedly paste a human shadow image as a static background into a forged image sequence, making the background and even the shadow completely identical in every frame. Under conventional comparison logic, due to the high pixel matching, the system would determine that the background has not changed. However, in real shooting, as long as the human body is breathing or experiencing slight shaking, the shadow cannot remain absolutely still at the pixel level. This makes forged images that violate the laws of physics easily pass detection. In summary, shadows cannot be simply categorized as foreground elements and eliminated, because they are attached to the background surface and change the visual appearance of the background, nor can they be directly compared as ordinary background textures. Therefore, how to mark the shadows to be tested that cannot establish a reliable correspondence with the shadow contour of the source image under the same detection mechanism, and then determine whether the mark originates from a reasonable deformation caused by drastic changes in human posture or is a real background tampering through subsequent background texture comparison, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0003] This invention provides a background tampering detection method based on human posture and shadow kinematic constraints, mainly including: The source image and the image to be tested are obtained as the basis for legality judgment. The global pixel distribution is extracted through a neural network to obtain human pose features, cast shadow features and global texture features. Based on the shadow pixel region marked by the shadow features, the geometric distribution parameters of the shadow in the source image are extracted. The geometric distribution parameters include the contour position and centroid position of the shadow pixel region. Based on the displacement components of the joints in the human posture features between the source image and the image to be tested, the theoretical spatial migration trajectory of the shadow in the source image under the posture of the image to be tested is determined. Based on the theoretical spatial migration trajectory and the displacement components of the joints in the human posture features, the kinematic relationship between the human posture features and the projected shadow features between the source image and the image to be tested is determined. Based on the kinematic correlation, the projected shadow features in the source image are geometrically transformed to obtain the predicted shadow features corresponding to the human posture in the image to be tested. The predicted shadow features are compared with the projected shadow features in the image to be tested to obtain the feature difference value. If the feature difference value meets the preset condition, a valid shadow mask is obtained; if the feature difference value does not meet the preset condition, an abnormal shadow mask is obtained. Based on the valid and abnormal shadow masks, mask regions are removed from the global texture features. The distribution of the remaining pixels at the same spatial location between the source image and the image under test is compared to determine the background tampering detection result of the image under test.

[0004] Furthermore, the acquisition of the source image and the image to be tested, which serve as the basis for legality determination, involves extracting features from the global pixel distribution using a neural network to obtain human pose features, cast shadow features, and global texture features, including: The source image and the image to be tested are normalized to the same resolution and spatially aligned in the pixel coordinate system. The color channels are separated into luminance and chrominance. An edge response layer is obtained by calculating the local gradient on the luminance component, and a color region layer is obtained by clustering the color difference between adjacent pixels on the chrominance component. The edge response layer and the color region layer are stacked along the channel dimension to obtain a multi-channel input tensor.

[0005] Furthermore, the acquisition of the source image and the image to be tested, which serve as the basis for legality determination, involves extracting features from the global pixel distribution using a neural network to obtain human pose features, cast shadow features, and global texture features, including: The source image and the image to be tested are fed into the backbone of the convolutional neural network. Multi-scale feature maps are obtained through layer-by-layer convolution and downsampling. The global feature tensor is obtained by upsampling and channel-by-channel stitching. The global feature tensor is divided into pose branch, shadow branch and background branch along the channel direction to obtain human pose features, cast shadow features and global texture features, respectively.

[0006] Furthermore, based on the shadow pixel region marked by the projected shadow features, the geometric distribution parameters of the projected shadow in the source image are extracted. These geometric distribution parameters include the contour position and centroid position of the shadow pixel region. Based on the displacement components of the joints in the human pose features between the source image and the image to be tested, the theoretical spatial migration trajectory of the projected shadow in the source image under the pose of the image to be tested is determined, including: A set of shadow contour sampling points is obtained by sampling at equal intervals along the outer contour of the shadow pixel region according to a closed curve; The centroid coordinates of the shadow are obtained by averaging the coordinates of all pixels within the shadow pixel region according to their positions. Based on the set of shadow contour sampling points, the contour shape similarity between the source image and the image to be tested is calculated. The contour shape similarity is measured by the shape context distance. The contour shape similarity is inversely proportional to the shape context distance. If the contour shape similarity is greater than the preset shape similarity threshold, then a one-to-one correspondence is performed according to the relative arc length along the contour, and the shadow centroid coordinates are directly paired to obtain shadow key point pairs. If the contour shape similarity is not greater than the preset shape similarity threshold, then shadow key point pairs cannot be established. The projected shadow features in the image to be tested are labeled as abnormal shadow masks. The steps of establishing kinematic correlation and predicting shadow features are skipped, and the process directly proceeds to the step of masking the global texture features based on the abnormal shadow mask and determining the background tampering test result.

[0007] Furthermore, the step of directly pairing the shadow centroid coordinates according to the relative arc length order along the contour to obtain shadow key point pairs includes: For each pair of shadow key points, a sparse displacement vector is obtained by subtracting the corresponding key point pixel coordinates in the source image from the key point pixel coordinates in the image to be tested. All sparse displacement vectors are arranged according to their source image coordinate positions to form a sparse displacement field. Two-dimensional thin-plate spline interpolation is used to extend the sparse displacement field to the unsampled pixel positions within the shadow pixel region, resulting in a dense displacement field covering all pixels. The theoretical spatial migration trajectory is obtained by arranging the fields along the space.

[0008] Furthermore, determining the kinematic correlation between the human posture features and projected shadow features between the source image and the image to be tested, based on the theoretical spatial migration trajectory and the displacement components of the joints in the human posture features, includes: The joint displacement vector is obtained by the difference between the two-dimensional coordinates of each joint point in the source image and the image to be tested. The directed line segment connecting adjacent joint points is denoted as the limb vector. The rotation component and scaling factor of the limb are obtained by taking the ratio of the angle between the limb vector of the source image and the limb vector of the test image. The translation component is obtained by taking the arithmetic mean of the joint displacement vector along the horizontal and vertical directions. The posture deformation descriptor is obtained by combining the translation component, the rotation component and the scaling factor. The shadow deformation descriptor is obtained by taking the overall translation, overall rotation angle and overall area change rate of the displacement vector of each pixel in the theoretical spatial migration trajectory. The linear mapping matrix from the attitude deformation descriptor to the shadow deformation descriptor is solved by least squares fitting. The coefficients of each row of the linear mapping matrix represent the contribution weight of each attitude deformation component to the corresponding shadow deformation component.

[0009] Furthermore, the step of performing a geometric transformation on the projected shadow features in the source image based on the kinematic correlation to obtain predicted shadow features corresponding to the human posture in the image to be tested includes: The pose deformation descriptor from the source image to the image under test is substituted into the linear mapping matrix in the kinematic correlation and right-multiplied to obtain the target shadow deformation descriptor containing the target translation component, the target rotation angle and the target area change rate. The target shadow deformation descriptor is then combined into a single geometric transformation matrix according to the synthesis rules of two-dimensional affine transformation. The coordinates of each pixel in the shadow pixel region of the source image are mapped according to the geometric transformation matrix to obtain the predicted shadow pixel scatter map.

[0010] Furthermore, the step of mapping the coordinates of each pixel within the shadow pixel region of the source image according to the geometric transformation matrix to obtain the predicted shadow pixel scatter map includes: For the unfilled pixel locations in the predicted shadow pixel scatter map that appear due to sparse mapping, a morphological closing operation is used to first expand along a preset structuring element and then erode along the same structuring element to obtain a predicted shadow mask that is internally continuous and has smooth boundaries. The predicted shadow mask is then used as the predicted shadow feature.

[0011] Furthermore, the process of comparing the predicted shadow features with the projected shadow features in the image to be tested to obtain a feature difference value, and if the feature difference value meets a preset condition, a valid shadow mask is obtained; if the feature difference value does not meet the preset condition, an abnormal shadow mask is obtained, including: Align the predicted shadow features and the projected shadow features in the same pixel coordinate system, take the intersection number and the union number of pixels for the pixel positions covered by the two shadow masks, subtract the intersection number from the union number of pixels and then divide by the union number of pixels to obtain the feature difference value that reflects the degree of spatial overlap between the two shadow masks; If the feature difference value does not exceed the preset deformation tolerance threshold, then the projected shadow feature is determined to conform to the kinematic correlation relationship, and the pixel position covered by the projected shadow feature is marked as a valid shadow mask; If the feature difference value exceeds the preset deformation tolerance threshold, the projected shadow feature is determined to be a lingering shadow detached from the human body posture, and the pixel position covered by the projected shadow feature is marked as an abnormal shadow mask.

[0012] Furthermore, the step of removing masked regions from global texture features based on valid and abnormal shadow masks, comparing the distribution of remaining pixels at the same spatial location between the source image and the image under test, and determining the background tampering detection result of the image under test includes: The pixel positions covered by at least one of the generated legal shadow masks and abnormal shadow masks are merged into a set of pixels to be removed. The pixel positions in the set of pixels to be removed are removed from the global texture features to obtain the remaining background pixel distribution of the source image and the remaining background pixel distribution of the image to be tested. The average difference of the remaining background is obtained by summing the absolute differences of the gray values ​​of the remaining background pixels at the same spatial location and then dividing by the total number of remaining pixel locations. If the average difference of the remaining background does not exceed a preset threshold, it is determined that the background has not been tampered with; otherwise, it is determined that the background has been tampered with.

[0013] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention discloses a background similarity detection method that integrates human body segmentation and adaptive background reconstruction. This method addresses the critical issue in video tampering detection where changes in human posture lead to altered projected shadows. However, tampered videos may contain outdated shadows or artificially added shadows that do not conform to lighting patterns. By extracting human posture features, projected shadow features, and global texture features through a neural network, a kinematic correlation is established between the spatial migration trajectory of shadow pixel regions and the displacement components of human joints. Based on this correlation, a geometric transformation is performed on the shadows in the source image to generate predicted shadow features. These predicted shadow features are compared with the actual shadows in the test image to obtain feature difference values, thereby distinguishing between legitimate shadows and lingering or fake shadows that are detached from the human posture. After generating an abnormal shadow mask, the masked regions of the global texture features are removed. Finally, background tampering detection is completed by comparing the distribution of remaining pixels at the same spatial location. This method effectively solves the technical problem that traditional methods cannot identify inconsistent shadow tampering, significantly improving the accuracy and robustness of video authenticity identification. Attached Figure Description

[0014] Figure 1 This is a flowchart of a background tampering detection method based on human posture and shadow kinematic constraints according to the present invention.

[0015] Figure 2 This is a schematic diagram of a background tampering detection method based on human posture and shadow kinematic constraints according to the present invention.

[0016] Figure 3 This is another schematic diagram of a background tampering detection method based on human posture and shadow kinematic constraints according to the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0018] like Figures 1-3 This embodiment of a background tampering detection method based on human posture and shadow kinematic constraints may specifically include: Step S101: Obtain the source image and the image to be tested as the basis for legality determination, and extract features of global pixel distribution through neural network to obtain human pose features, cast shadow features and global texture features.

[0019] The source image and the image to be tested are acquired and normalized to the same resolution to ensure spatial alignment in the pixel coordinate system. Luminosity and chromaticity are separated for the color channels of the source and test images. An edge response layer is calculated based on the local gradient of the luminance component, and a color region layer is obtained by clustering adjacent pixel color differences for the chromaticity component. The edge response layer and the color region layer are stacked along the channel dimension to obtain a multi-channel input tensor. This multi-channel input tensor is fed into the backbone of a pre-trained convolutional neural network. Multi-scale feature maps are obtained through layer-by-layer convolution and downsampling. Upsampling and channel-by-channel concatenation are then used to fuse these multi-scale feature maps to obtain a fused global feature tensor. A three-branch decoding method is used for this global feature tensor, dividing it along the channel direction into a pose branch, a shadow branch, and a background branch. In the pose branch, the joint coordinates are located by taking the local maxima of the joint heatmap to obtain the human pose features; in the shadow branch, the low-brightness connected regions are masked and extracted in combination with the relative positions of the human pose features to obtain the projected shadow features; in the background branch, the remaining pixels after removing the human body region and the shadow region are combined with the color region layer to perform texture aggregation to obtain the global texture features.

[0020] The source image serves as the legal baseline image, and the image to be tested serves as the image to be reviewed. Both enter the feature extraction stage together, ultimately separating three independent layers: human posture features, projected shadow features, and global texture features.

[0021] In one embodiment, the source image and the image to be tested are normalized to the same resolution.

[0022] For example, the longer sides of both images are uniformly scaled to 512 pixels, and the shorter sides are scaled according to the original aspect ratio. Then, the remaining areas are filled with zero pixels, so that the two images are aligned point by point in the pixel coordinate system. The purpose of this spatial alignment is to ensure that the same coordinate position in subsequent comparisons points to the same area in the real physical space.

[0023] Specifically, the normalized image employs a luminance and chrominance channel separation. The red, green, and blue components of each pixel are converted to obtain the luminance component Y using the formula Y = 0.299R + 0.587G + 0.114B, where R, G, and B represent the red, green, and blue intensity values ​​of the pixel, respectively. The remaining chrominance components are denoted as U and V. The luminance component Y carries information about illumination changes and shadow grayscale, while the chrominance components U and V carry the inherent color information of the background object. For each pixel, the difference between adjacent pixels is calculated in both the horizontal and vertical directions based on the luminance component Y. The square root of the sum of the squares of these differences yields the gradient magnitude of that pixel. All gradient magnitudes are aggregated into an edge response layer, which is used to delineate object contours and shadow boundaries.

[0024] Preferably, nearest neighbor clustering is used for the chromaticity components U and V. Pixels with a U and V difference less than a preset color clustering threshold are grouped into the same color region, and all color regions form a color region layer. The edge response layer and the color region layer are stacked along the channel dimension after the luminance component to obtain a multi-channel input tensor. The difference between the U and V chromaticity components ranges from 0 to 255, and the preset color clustering threshold ranges from 5 to 20. This value is determined by analyzing the chromaticity fluctuation range of the same object surface under different lighting conditions. The smaller the value, the more sensitive it is to color differences, and the more subdivided the generated color regions are. The larger the value, the higher the degree of color region merging. The specific value can be adaptively adjusted according to the uniformity of lighting and the color detection sensitivity requirements in the actual scene.

[0025] Specifically, the multi-channel input tensor is fed into a pre-trained convolutional neural network (CNN) backbone. The CNN backbone employs a residual connection structure, where the output of each convolutional layer is bitwise added to its input before being passed to the next layer. Forward gradients are propagated along skip paths, avoiding gradient decay during backpropagation in deep networks. The CNN backbone outputs feature maps with different spatial resolutions at different levels; shallow feature maps retain pixel-level detail responses, while deep feature maps retain region-level semantic responses. In one implementation, the multi-scale feature maps are fused using bilinear upsampling and channel-wise concatenation. Deep feature maps are enlarged along the spatial dimension to the same size as shallow feature maps and then concatenated end-to-end along the channel dimension to obtain the fused global feature tensor. This global feature tensor simultaneously carries pixel-level detail responses and region-level semantic responses. Furthermore, the global feature tensor is divided along the channel direction into pose, shadow, and background branches. These three branches share the parameters of the CNN backbone, and each branch independently completes the decoding of its corresponding category. In the pose branch, the global feature tensor is upsampled through several layers of deconvolution to restore it to the same spatial resolution as the original image, resulting in a joint heatmap. The number of channels in the joint heatmap is the same as the number of predefined joint categories, with each channel corresponding to one joint category.

[0026] Specifically, the joint categories include head, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. For each channel, a local maximum is taken in the spatial dimension; the location of this local maximum is the two-dimensional coordinate of that joint. All joint coordinates are connected end-to-end according to a preset limb connection order to obtain the human posture features. In the shadow branch, the probability value of each pixel belonging to the projected shadow is obtained by regression of the global feature tensor. Pixels with a probability value higher than a preset candidate shadow probability threshold are marked as candidate shadow pixels. The candidate shadow probability threshold ranges from 0.3 to 0.7. This value is obtained through statistical analysis of typical surge arrester transient voltage waveforms. In real impact events, the probability confidence interval between the waveform peak and the noise baseline is usually between 0.4 and 0.6. Using 0.3 to 0.7 as the candidate range can cover most effective pulse signals while effectively eliminating spurious pulses caused by background noise. The specific value can be adaptively adjusted according to the on-site noise level and signal acquisition sensitivity requirements.

[0027] It should be noted that the candidate shadow pixels need to be further filtered based on the relative position of the human posture features. Only candidate pixels located in the ground area around the foot coordinates of the human posture features and the wall area corresponding to the back coordinates are retained. Environmental shadows cast by fixed objects in the image are removed. The filtered set of connected pixels is used as the projected shadow feature. In the background branch, the areas covered by the human posture features and the areas covered by the projected shadow features are removed based on the global feature tensor. The remaining pixels are combined with the aforementioned color region layer for texture consistency aggregation. Adjacent pixels whose color and texture statistics differ by less than a preset texture aggregation threshold are grouped into the same background segment. All background segments are combined to obtain the global texture feature. The value range of the texture aggregation threshold is determined based on the gray-level variance of the local image region. For images normalized to the gray-level range of 0-255, the threshold is usually set to 10-30. The smaller the value, the more similar the texture features of adjacent pixels are required for merging, resulting in a more refined background segment. The larger the value, the greater the texture difference is allowed, resulting in a larger background area. The specific value can be adaptively adjusted according to the resolution of the image to be detected and the complexity of the background texture.

[0028] It is understood that the above three-branch decoding is performed on the source image and the image to be tested respectively, and finally each of them obtains a human pose feature, a projected shadow feature and a global texture feature. The three feature layers do not overlap and their spatial positions correspond one-to-one, serving as a unified input for subsequent shadow migration trajectory modeling and background comparison.

[0029] Step S102: Based on the shadow pixel region marked by the projected shadow features, determine the shadow pixel region in the source image and the shadow pixel region in the image to be tested, determine the pixel displacement vector from the shadow pixel region of the source image to the shadow pixel region of the image to be tested, and obtain the theoretical spatial migration trajectory.

[0030] Based on the shadow pixel region marked by the projected shadow features in the source image, sampling is performed at equal intervals along the outer contour of the shadow pixel region using a closed curve to obtain a set of shadow contour sampling points in the source image. The average of all pixel coordinates within the shadow pixel region is then used to obtain the centroid coordinates of the shadow in the source image. The same equal-interval sampling rule is applied to the projected shadow features in the image under test to obtain a set of shadow contour sampling points and centroid coordinates for the image under test. The extracted contour sampling point sets and centroid coordinates of the source image and the image under test constitute geometric distribution parameters. The set of shadow contour sampling points in the source image and the set of shadow contour sampling points in the image under test are matched one-to-one according to the relative arc length of the contour. The centroid coordinates of the shadow in the source image and the centroid coordinates of the shadow in the image under test are directly paired to obtain shadow keypoint pairs. For each shadow keypoint pair, the pixel coordinates of the keypoint in the source image are subtracted from the pixel coordinates of the corresponding keypoint in the image under test to obtain a sparse displacement vector at that keypoint. All sparse displacement vectors are arranged according to their coordinate positions in the source image to form a sparse displacement field. Two-dimensional thin-plate spline interpolation is used for the sparse displacement field to extend the sparse displacement vector to the unsampled pixel positions in the shadow pixel region, thereby obtaining a dense displacement field that covers all pixels in the shadow pixel region. The displacement vector recorded at each pixel position in the dense displacement field is arranged along the space to obtain the theoretical spatial migration trajectory.

[0031] In one embodiment, a closed curve is extracted along the outer contour of the shadow pixel region marked by the projected shadow feature in the source image.

[0032] Specifically, the boundary pixels of the shadow pixel region are concatenated according to their adjacent relationships to obtain a boundary pixel chain that is connected end to end; the boundary pixel chain is sampled at equal intervals according to the arc length, and a pixel position is taken as a contour sampling point every preset arc length interval.

[0033] Preferably, 1 / 64 of the total length of the boundary pixel chain is taken at equal intervals to obtain 64 contour sampling points. 64 sampling points are sufficient to characterize the macroscopic geometric features of the shadow contour, while maintaining a moderate computational complexity. This number can be appropriately adjusted in practical applications according to the image resolution and the complexity of the shadow contour. All contour sampling points constitute the source image shadow contour sampling point set.

[0034] Specifically, the arithmetic mean of the coordinates of all pixels within the shadow pixel region is taken by calculating the arithmetic mean of the horizontal and vertical coordinates. The resulting two-dimensional coordinates are the centroid coordinates of the shadow in the source image. The centroid coordinates reflect the overall position center of the shadow pixel region and, together with the contour sampling points, cover both the boundary and center positions of the shadow pixel region.

[0035] It should be noted that the same equidistant sampling rules and centroid calculation rules are used for the projected shadow features in the image under test as for the source image, to obtain the set of shadow contour sampling points and the centroid coordinates of the shadow in the image under test, ensuring that the key points extracted from the two images correspond one-to-one in terms of quantity and sampling rules. Furthermore, the arc length order of the set of shadow contour sampling points in the source image and the set of shadow contour sampling points in the image under test is determined.

[0036] For example, contour sampling points are numbered sequentially from the starting position of the outer contour of the shadow pixel region in the source image along the closed curve in a clockwise direction, from point 1 to point 64. The same starting position determination rule is applied to the shadow pixel region of the image under test, and contour sampling points are also numbered sequentially from point 1 to point 64 along the closed curve in a clockwise direction. The starting position determination rule takes the uppermost and leftmost boundary pixel of the shadow pixel region as the starting point for numbering. Contour sampling points with the same number in the two images constitute a pair of shadow contour keypoints. The centroid coordinates of the shadow in the source image and the centroid coordinates of the shadow in the image under test are directly paired to obtain a pair of shadow centroid keypoints. All contour keypoint pairs and centroid keypoint pairs are combined to form a pair of shadow keypoints.

[0037] Understandably, for each pair of shadow keypoints, the pixel coordinates of the keypoint in the source image are subtracted from the pixel coordinates of the corresponding keypoint in the image under test to obtain a two-dimensional difference along the horizontal and vertical directions at that keypoint. This two-dimensional difference serves as the sparse displacement vector at that keypoint. All sparse displacement vectors are arranged according to their coordinate positions in the source image to obtain a sparse displacement field, which has values ​​only at 65 keypoint positions.

[0038] Specifically, two-dimensional thin-plate spline interpolation is used for the sparse displacement field. This two-dimensional thin-plate spline interpolation is an interpolation method based on minimizing the elastic deformation energy of a thin plate. It treats the sparsely distributed displacement vectors as known displacements of a thin plate at several control points, and solves for the displacements of the entire thin plate at all unknown positions, thus minimizing the bending energy of the thin plate. Specifically, let the coordinates of the 65 control points of the sparse displacement field be (x... i y i The corresponding known displacement vector is (dx) i dy i For any position to be interpolated (x, y), the functional form of two-dimensional thin-plate spline interpolation is: When r > 0, the basis function U(r) = r 2 lnr, where r is the Euclidean distance from the location to be interpolated to the control point, r=(x x i ) 2 +(y y i )2 a1, a2, a3 are affine transformation parameters, w i Let be the weighting coefficient for the i-th control point. After determining the affine parameters and weighting coefficients by solving the linear equations of the thin-plate spline interpolation, the displacement vector at any interpolation location can be obtained by substituting the coordinates into the above function. The bending energy is composed of the square integrals of the second-order partial derivatives of the displacement function in the transverse and longitudinal directions. The thin-plate spline interpolation precisely passes through the given displacement at all control point locations, and smoothly transitions between control points according to the principle of minimizing bending energy.

[0039] Preferably, for the unsampled pixel positions within the shadow pixel region of the source image, the displacement vector at that pixel position is solved using the two-dimensional thin-plate spline interpolation, resulting in a dense displacement field covering all pixels in the shadow pixel region. The displacement vectors recorded at each pixel position in the dense displacement field are arranged sequentially along the spatial position, which constitutes the theoretical spatial migration trajectory. The theoretical spatial migration trajectory depicts the point-to-point migration path of each pixel in the shadow pixel region of the source image to the corresponding pixel in the shadow of the image under test.

[0040] It is understandable that after obtaining the sets of shadow contour sampling points for the source image and the image to be tested, the contour shape similarity is calculated based on the two sets of contour sampling points. The contour shape similarity uses the shape context distance metric, and the contour shape similarity is inversely proportional to the shape context distance. Specifically, for sampling point p on the contour of the source image and sampling point q on the contour of the image to be tested, shape context descriptors h are constructed respectively. p with h q The space is divided into 5 radius intervals and 12 angle intervals, totaling 60 sectors. The distribution of sampling points within each sector is statistically analyzed as a descriptor. The matching cost between point p and point q is calculated using the χ² distance. Where k is the sector index, h p (k) and h q (k) represents the normalized sampling point counts of points p and q in the kth sector, respectively. The overall shape distance between the two contours is calculated using the Hungarian algorithm to find the optimal match between all sampling points, minimizing the total matching cost. This minimum total matching cost is the shape context distance. If the contour shape similarity is greater than the preset shape similarity threshold, it indicates that the shadow contours of the two images have sufficient geometric correspondence, and the subsequent arc length order correspondence and key point pairing steps are continued. If the contour shape similarity is not greater than the preset shape similarity threshold, it indicates that a reliable pixel-level correspondence cannot be established between the shadow contours of the source image and the image to be tested. In this case, the projected shadow features in the image to be tested are directly labeled as abnormal shadow masks, and the subsequent theoretical spatial migration trajectory calculation, kinematic correlation establishment, and shadow feature prediction steps are no longer performed. Instead, the process jumps directly to step S106 to perform the background texture comparison step.

[0041] Specifically, the preset shape similarity threshold corresponds to the upper limit of the allowable deviation of the shape context distance, and its value ranges from 0.5 to 0.9. The specific value can be adjusted according to the requirements of deformation sensitivity in the actual application scenario. The higher the value, the more similar the shadow contours of the source image and the image to be tested are required to continue to perform subsequent kinematic modeling steps. Conversely, a smaller value allows for greater contour differences.

[0042] Step S103: Based on the theoretical spatial migration trajectory and the displacement components of the joints in the human posture features, determine the kinematic relationship between the human posture features and the projected shadow features between the source image and the image to be tested.

[0043] Based on the two-dimensional coordinate difference between each joint point in the source image and the image under test in the human posture features, the joint displacement vector of each joint point is obtained by subtracting the coordinates of the source image from the coordinates of the image under test. The directed line segment connecting adjacent joint points is used as the limb vector, and the limb vectors of the source image and the image under test are obtained in the two images respectively. The rotation component and scaling factor of the limb are obtained by taking the ratio of the angle and length between the limb vectors of the source image and the limb vectors of the image under test. The translation component of the human body is obtained by taking the arithmetic mean of the joint displacement vectors along the horizontal and vertical directions. The translation component, the rotation component, and the scaling factor are combined to form a posture deformation descriptor. The posture deformation descriptor has 27 dimensions, where the translation component consists of two dimensions: horizontal and vertical translation. Each of the 13 limb segments corresponds to one rotation component and one scaling factor, for a total of 26 dimensions, which are concatenated in the order of "translation component—rotation component of each limb—scaling factor of each limb". The posture deformation descriptor describes the complete deformation features of the human posture from the source image to the image under test. For each pixel displacement vector in the theoretical spatial migration trajectory, the overall translation, overall rotation angle, and overall area change rate are taken in the same way to obtain the shadow deformation descriptor. For the values ​​of the corresponding components of the attitude deformation descriptor and the shadow deformation descriptor, the least squares fitting method is used to solve for the linear mapping matrix from the attitude deformation descriptor to the shadow deformation descriptor. Let the attitude deformation descriptor be a 27-dimensional column vector x, the shadow deformation descriptor be a 3-dimensional column vector y, and the linear mapping matrix W be a 3x27 matrix to be solved, satisfying y≈W×x. Multiple sets of corresponding attitude deformation descriptor and shadow deformation descriptor sample pairs {(x1, y1), (x2, y2), ..., (x... n y n Construct the objective function minΣ i ||W×x i -y i ||², taking the derivative of the objective function with respect to W and setting the derivative to zero, we obtain the normal equation W=(Y×X). )×(X×X ) -1 Where X is a 27-row, n-column sample independent variable matrix and Y is a 3-row, n-column sample dependent variable matrix, the linear mapping matrix W can be obtained by matrix inversion. When the number of samples is large, singular value decomposition can be used instead of direct inversion to improve numerical stability. The coefficients of each row of the linear mapping matrix represent the contribution weight of the corresponding shadow deformation component to each posture deformation component, and all contribution weights constitute the kinematic relationship between human posture features and projected shadow features.

[0044] In one implementation, joint displacement vectors are calculated one by one for each of the 13 joints in the human posture features.

[0045] Specifically, for each joint point, its two-dimensional coordinates in the image to be tested and its two-dimensional coordinates in the source image are taken. The two-dimensional difference between the source coordinates and the measured coordinates is obtained by subtracting the measured coordinates from the measured coordinates. The two-dimensional difference is the joint displacement vector of that joint point. There are 13 joint points corresponding to 13 joint displacement vectors. All joint displacement vectors are arranged in order according to their respective joint categories to form a joint displacement vector sequence.

[0046] Specifically, for a predefined limb connection order, adjacent joints are connected by a directed line segment to form a limb vector. For example, the left shoulder to the left elbow forms the left upper arm limb vector, and the left elbow to the left wrist forms the left forearm limb vector. The source image limb vector set and the test image limb vector set are obtained from the two images according to the same connection order. Further, the rotation component and scaling factor are solved segment by segment for the source image limb vector and the test image limb vector.

[0047] In one embodiment, the pointing angle of the endpoint coordinate difference of the limb vector in the two images is recorded on the two-dimensional plane. The pointing angle of the limb vector in the image to be tested is subtracted from the pointing angle of the limb vector in the source image to obtain the rotation component of the limb segment between the two images.

[0048] Preferably, the rotation component is limited to a range of -180 degrees to 180 degrees, and values ​​outside this range are periodically folded back into this range. The magnitude of the limb vector in each of the two images is calculated by taking the square root of the sum of the squares of the differences between the coordinates of the two endpoints. The magnitude of the limb vector in the image under test is divided by the magnitude of the limb vector in the source image to obtain the scaling factor of the limb segment between the two images. A scaling factor greater than 1 indicates that the limb is relatively elongated in the image, and less than 1 indicates that the limb is relatively shortened. Each of the 13 limb segments has its own rotation component and scaling factor, which represent the angular and length deformation of each limb segment.

[0049] It should be noted that the arithmetic mean of the joint displacement vector sequence is taken along the horizontal direction and the arithmetic mean is taken along the vertical direction to obtain a two-dimensional average vector as the translation component of the human body. The translation component represents the displacement of the entire human body in the image.

[0050] It is understood that the translation component, the rotation component, and the scaling factor are concatenated into a one-dimensional numerical sequence in a prescribed order to form a posture deformation descriptor. This posture deformation descriptor characterizes the complete deformation features of the human posture from the source image to the image under test.

[0051] Specifically, shadow deformation descriptors are extracted for the theoretical spatial migration trajectory using the same stitching rules. The overall shadow translation component is obtained by taking the arithmetic mean of the horizontal and vertical displacement vectors of all pixels in the theoretical spatial migration trajectory.

[0052] In one embodiment, for the vector pairs formed by the displacement vectors of each pixel in the theoretical spatial migration trajectory and the relative displacement of that pixel to the centroid coordinates of the shadow in the source image, the angle closest to the latter endpoint of the displacement vector after rotating the relative displacement vector around the centroid coordinates of the shadow is taken as the overall shadow rotation angle; the overall shadow area change rate is obtained by the ratio of the total number of pixels in the shadow pixel region of the test image to the total number of pixels in the shadow pixel region of the source image. The overall shadow translation component, the overall shadow rotation angle, and the overall shadow area change rate are concatenated into a one-dimensional numerical sequence in the same order as the attitude deformation descriptor to form the shadow deformation descriptor. Further, for the values ​​of the attitude deformation descriptor and the shadow deformation descriptor on the corresponding components, the linear mapping matrix from the attitude deformation descriptor to the shadow deformation descriptor is solved by least squares fitting. The least squares fitting uses attitude deformation descriptors as the independent variable column and shadow deformation descriptors as the dependent variable column. It solves for the matrix that minimizes the sum of squares of the differences between the dependent variable column and the independent variable column multiplied by the matrix to be solved. The matrix to be solved is the linear mapping matrix.

[0053] It is understood that the coefficients of each row of the linear mapping matrix represent the contribution weight of each posture deformation component to the corresponding shadow deformation component. All contribution weights together constitute the kinematic relationship between human posture features and projected shadow features, and the kinematic relationship describes the quantitative mapping relationship between human posture deformation and projected shadow deformation.

[0054] It should be noted that if the projected shadow features of the image to be tested have been identified as an abnormal shadow mask in step S102 and the process has jumped to step S106, then this step will not be executed.

[0055] Step S104: Based on the kinematic correlation, perform a geometric transformation on the projected shadow features in the source image to obtain the predicted shadow features corresponding to the human posture in the image to be tested.

[0056] Based on the linear mapping matrix in the kinematic correlation, the pose deformation descriptor from the source image to the image under test is substituted into the linear mapping matrix and right-multiplied to obtain the target shadow deformation descriptor corresponding to the pose of the image under test. The target shadow deformation descriptor includes the target translation component, the target rotation angle, and the target area change rate. The target translation component, the target rotation angle, and the target area change rate are combined into a single geometric transformation matrix according to the synthesis rules of two-dimensional affine transformation. Based on the geometric transformation matrix, the coordinates of each pixel within the shadow pixel region marked by the projected shadow features in the source image are right-multiplied to obtain the target coordinates of that pixel in the coordinate system of the image under test. After all pixel coordinates are mapped in this way, the grayscale value of each mapped point is moved to the nearest integer pixel position to obtain the predicted shadow pixel scatter map. For the unfilled pixel locations in the predicted shadow pixel scatter map that appear due to sparse mapping, morphological closing operation is used to fill the holes and smooth the boundaries. The morphological closing operation first expands along a preset structuring element and then erodes along the same structuring element to obtain a predicted shadow mask that is internally continuous and has smooth boundaries. The predicted shadow mask serves as the predicted shadow feature corresponding to the human pose in the image to be tested.

[0057] In one implementation, the attitude deformation descriptor from the source image to the image under test is right-multiplied by the linear mapping matrix in column vector form. The coefficients of each row of the linear mapping matrix are multiplied element-wise with each component of the attitude deformation descriptor and then summed to obtain a new column vector, which is the target shadow deformation descriptor.

[0058] Specifically, the target shadow deformation descriptor includes three components: target translation component, target rotation angle, and target area change rate.

[0059] It is understood that the target translation component, the target rotation angle, and the target area change rate need to be combined into a geometric transformation matrix that can directly affect the pixel coordinates. The synthesis rule of the two-dimensional affine transformation is a general convention in the field of image geometry processing, which stipulates that the pixel coordinates are rotated first, then scaled, and finally translated.

[0060] Specifically, the rotation matrix is ​​arranged in the standard form of a two-dimensional rotation matrix, consisting of the cosine and sine values ​​of the target rotation angle; the scaling matrix is ​​composed of the square root of the target area change rate, which serves as the horizontal and vertical scaling ratios, placed diagonally; the translation vector consists of the two components of the target translation component along the horizontal and vertical axes. The rotation matrix and the scaling matrix are multiplied sequentially using matrix multiplication, and then the translation vector is added to synthesize a single 3x3 geometric transformation matrix in homogeneous coordinates.

[0061] It should be noted that, for the homogeneous coordinate form in the geometric transformation matrix, the third row is fixed at 0, 0, and 1, while the six values ​​in the first two rows respectively carry the coupling coefficients of rotation and scaling, as well as the horizontal and vertical translation amounts. Further, for each pixel coordinate within the shadow pixel region marked by the projected shadow features in the source image, the pixel coordinate, along with the additional constant 1, forms a 3D homogeneous coordinate column vector. This homogeneous coordinate column vector is then right-multiplied by the geometric transformation matrix. The first two components of the resulting column vector are the target coordinates of that pixel in the coordinate system of the image under test. All pixel coordinates within the shadow pixel region are mapped point-by-point according to the same rule.

[0062] Specifically, for each mapped target coordinate, it is rounded to the nearest integer pixel position, and the grayscale value of that pixel in the source image is moved to that integer pixel position. After all pixel grayscale values ​​have been moved according to the above rules, a predicted shadow pixel scatter map is obtained. The predicted shadow pixel scatter map contains unfilled pixel positions due to uneven distribution of target coordinates.

[0063] Preferably, morphological closing operations are applied to the predicted shadow pixel scatter map. The morphological closing operation includes two sub-operations: dilation and erosion. The preset structuring element is a disk-shaped structure with a radius of 3 pixels. Specifically, the binary image set A represents the set of currently filled pixel positions, and B represents the set of disk-shaped structuring elements with a radius of 3 pixels. The mathematical definition of the dilation operation is: B r Let B be a symmetric set about the origin. Indicates that B r Translate to position z. The dilation operation marks all positions within the structuring element's coverage area that have at least one filled pixel as filled. The mathematical definition of the erosion operation is: ,in The erosion operation shifts structuring element B to position z. The erosion operation retains the positions where the structuring element completely falls into the filled area as filled, and marks the remaining positions as unfilled. The dilation operation, for each filled pixel position in the predicted shadow pixel scatter map, marks all pixel positions covered by the surrounding disk structure as filled. The erosion operation, for the dilated pixel distribution, re-marks the positions where there are unfilled pixels within the area covered by the surrounding disk structure as unfilled.

[0064] It is understood that after the dilation and erosion operations are performed sequentially, a predicted shadow mask with continuous internal structure and smooth boundaries is obtained. The predicted shadow mask identifies the shadow pixel region corresponding to the human pose in the image under test, which is the predicted shadow feature corresponding to the human pose in the image under test.

[0065] It should be noted that if the projected shadow features of the image to be tested have been identified as an abnormal shadow mask in step S102 and the process has jumped to step S106, then this step will not be executed.

[0066] Step S105: Compare the predicted shadow features with the projected shadow features in the image to be tested to obtain a feature difference value. If the feature difference value meets the preset condition, it is determined that the projected shadow features in the image to be tested conform to the kinematic correlation relationship and a valid shadow mask is obtained. If the feature difference value does not meet the preset condition, it is determined that the projected shadow features in the image to be tested belong to a lingering shadow that is detached from the human posture and an abnormal shadow mask is obtained.

[0067] Based on the predicted shadow features and the projected shadow features in the image under test, the two shadow masks are aligned in the same pixel coordinate system. The intersection and union of the pixel positions covered by the two shadow masks are calculated. The intersection pixel count is subtracted from the union pixel count, and then divided by the union pixel count to obtain a feature difference value reflecting the degree of spatial overlap between the two shadow masks. This feature difference value is compared with a preset deformation tolerance threshold. If the feature difference value does not exceed the preset deformation tolerance threshold, the projected shadow features in the image under test are determined to conform to the kinematic correlation, and the pixel positions covered by the projected shadow features in the image under test are labeled as valid shadow masks. If the feature difference value exceeds the preset deformation tolerance threshold, the projected shadow features in the image under test are determined to be persistent shadows detached from the human posture, and the pixel positions covered by the projected shadow features in the image under test are labeled as abnormal shadow masks.

[0068] In one implementation, for the predicted shadow features and the projected shadow features in the image to be tested, the two shadow masks are aligned to the same pixel coordinate system according to their respective pixel positions.

[0069] Specifically, for each pixel position, it is determined whether it falls within the area covered by both shadow masks. Pixels falling within the area covered by both masks are grouped into the intersection pixel set, and pixels falling within the area covered by either mask are grouped into the union pixel set. The number of pixels in the union pixel set minus the number of pixels in the intersection pixel set, and then divided by the number of pixels in the union pixel set, yields a feature difference value reflecting the degree of spatial overlap between the two shadow masks. This feature difference value falls between 0 and 1.

[0070] It should be noted that the preset deformation tolerance threshold represents the deformation tolerance allowed by the kinematic correlation.

[0071] Preferably, one embodiment of the preset deformation tolerance threshold is set to 0.3. This value is obtained through experimental statistics on a large number of real-life human posture change videos. When the human posture changes continuously, the difference in the intersection-union ratio (IU) between the kinematically predicted mask and the actual shadow mask for the same shadow area typically fluctuates between 0.05 and 0.25. Setting 0.3 as the upper limit of tolerance can cover most legitimate posture change scenarios, while effectively identifying fake shadows with significantly higher IU values ​​(greater than 0.4). This threshold can be adjusted according to the trade-off between detection sensitivity and false alarm rate in actual application scenarios. Lowering the threshold will increase the detection sensitivity for fake shadows but may increase the false alarm rate, while raising the threshold will have the opposite effect. If the feature difference value does not exceed the preset deformation tolerance threshold, the projected shadow feature in the image under test is determined to conform to the kinematic correlation, and the pixel position covered by the projected shadow feature in the image under test is labeled as a valid shadow mask; if the feature difference value exceeds the preset deformation tolerance threshold, the projected shadow feature in the image under test is determined to be a lingering shadow detached from the human posture, and the pixel position covered by the projected shadow feature in the image under test is labeled as an abnormal shadow mask. The valid shadow mask and the abnormal shadow mask respectively carry two different sets of pixel positions: shadow regions matching the human posture and shadow regions violating the human posture.

[0072] It should be noted that if the projected shadow features of the image to be tested have been identified as an abnormal shadow mask in step S102 and the process has jumped to step S106, then this step will not be executed.

[0073] Step S106: Based on the valid shadow mask and the abnormal shadow mask, the global texture features are masked and removed. Then, the distribution of the remaining pixels at the same spatial position between the source image and the image to be tested is compared to determine the background tampering test result of the image to be tested.

[0074] Based on the pixel positions marked by the valid shadow mask and the abnormal shadow mask, the pixel positions covered by at least one type of the generated valid shadow mask and abnormal shadow mask are merged into a set of pixels to be removed. The global texture features in the source image and the image under test are scanned pixel by pixel, and the pixel positions falling within the set of pixels to be removed are removed from the global texture features, resulting in the remaining background pixel distribution of the source image and the remaining background pixel distribution of the image under test. The absolute difference between the brightness component Y values ​​at the same spatial position of the remaining background pixel distribution of the source image and the remaining background pixel distribution of the image under test is taken. All absolute differences are summed at the remaining pixel positions and divided by the total number of remaining pixel positions to obtain the average difference of the remaining background. If the average difference of the remaining background does not exceed a preset threshold, the background tampering test result of the image under test is determined to be untampered; if the average difference of the remaining background exceeds the preset threshold, the background tampering test result of the image under test is determined to be tampered.

[0075] In one implementation, a union is performed on the pixel positions marked by the valid shadow mask and the abnormal shadow mask.

[0076] Specifically, for each pixel position in the image, it is determined whether it falls within the coverage area of ​​a valid shadow mask or an abnormal shadow mask. If any mask covers the pixel position, the pixel position is added to the set of pixels to be removed.

[0077] Preferably, the global texture features in the source image and the image to be tested are scanned pixel by pixel, and the pixel positions that fall within the set of pixels to be removed are removed from the global texture features to obtain the remaining background pixel distribution of the source image and the remaining background pixel distribution of the image to be tested.

[0078] It is understood that, for pixels at the same spatial location in the remaining background pixel distribution of the source image and the remaining background pixel distribution of the image under test, the absolute difference in grayscale value is extracted point by point from the value of the luminance component Y obtained by the aforementioned conversion Y=0.299R+0.587G+0.114B. This absolute difference in grayscale value represents the degree of brightness deviation between the two images at the same coordinate position. The average difference in grayscale value is obtained by summing the absolute differences in grayscale value at all remaining pixel positions and dividing by the total number of remaining pixel positions.

[0079] In one embodiment, the preset threshold is set to 8, with a grayscale level ranging from 0 to 255. This value is obtained through statistical analysis of various common shooting devices, including mobile phone cameras, webcams, and professional cameras, and through empty background image sequences under typical indoor and outdoor lighting conditions. Between two consecutively captured images of the same scene, the average pixel grayscale difference caused by sensor noise and minor fluctuations in lighting is typically between 2 and 6, with a maximum difference usually not exceeding 8. Using 8 as the upper tolerance limit effectively distinguishes between natural fluctuations in actual shooting and significant pixel differences resulting from background replacement, the latter typically exceeding 20. This threshold can be adaptively adjusted according to the noise level of the image acquisition device and the lighting stability requirements of the application scenario. If the average difference of the remaining background does not exceed the preset threshold, the background tampering test result of the image under test is determined to be unaltered; if the average difference of the remaining background exceeds the preset threshold, the background tampering test result of the image under test is determined to be tampered with.

[0080] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A background tampering detection method based on human posture and shadow kinematic constraints, characterized in that, The method includes: The source image and the image to be tested are obtained as the basis for legality judgment. The global pixel distribution is extracted through a neural network to obtain human pose features, cast shadow features and global texture features. Based on the shadow pixel region marked by the shadow features, the geometric distribution parameters of the shadow in the source image are extracted. The geometric distribution parameters include the contour position and centroid position of the shadow pixel region. Based on the displacement components of the joints in the human posture features between the source image and the image to be tested, the theoretical spatial migration trajectory of the shadow in the source image under the posture of the image to be tested is determined. Based on the theoretical spatial migration trajectory and the displacement components of the joints in the human posture features, the kinematic relationship between the human posture features and the projected shadow features between the source image and the image to be tested is determined. Based on the kinematic correlation, the projected shadow features in the source image are geometrically transformed to obtain the predicted shadow features corresponding to the human posture in the image to be tested. The predicted shadow features are compared with the projected shadow features in the image to be tested to obtain the feature difference value. If the feature difference value meets the preset condition, a valid shadow mask is obtained; if the feature difference value does not meet the preset condition, an abnormal shadow mask is obtained. Based on the valid and abnormal shadow masks, mask regions are removed from the global texture features. The distribution of the remaining pixels at the same spatial location between the source image and the image under test is compared to determine the background tampering detection result of the image under test.

2. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 1, characterized in that, The process involves acquiring the source image and the image to be tested, which serve as the basis for legality determination. Then, a neural network is used to extract features from the global pixel distribution, resulting in human pose features, cast shadow features, and global texture features, including: The source image and the image to be tested are normalized to the same resolution and spatially aligned in the pixel coordinate system. The color channels are separated into luminance and chrominance. An edge response layer is obtained by calculating the local gradient on the luminance component, and a color region layer is obtained by clustering the color difference between adjacent pixels on the chrominance component. The edge response layer and the color region layer are stacked along the channel dimension to obtain a multi-channel input tensor.

3. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 1, characterized in that, The process involves acquiring the source image and the image to be tested, which serve as the basis for legality determination. Then, a neural network is used to extract features from the global pixel distribution, resulting in human pose features, cast shadow features, and global texture features, including: The source image and the image to be tested are fed into the backbone of the convolutional neural network. Multi-scale feature maps are obtained through layer-by-layer convolution and downsampling. The global feature tensor is obtained by upsampling and channel-by-channel stitching. The global feature tensor is divided into pose branch, shadow branch and background branch along the channel direction to obtain human pose features, cast shadow features and global texture features, respectively.

4. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 1, characterized in that, The process involves extracting geometric distribution parameters of the projected shadow in the source image based on the shadow pixel region identified by the projected shadow features. These geometric distribution parameters include the contour position and centroid position of the shadow pixel region. Furthermore, based on the displacement components of the joints in the human pose features between the source image and the image under test, the theoretical spatial migration trajectory of the projected shadow in the source image under the pose of the image under test is determined. This includes: A set of shadow contour sampling points is obtained by sampling at equal intervals along the outer contour of the shadow pixel region according to a closed curve; The centroid coordinates of the shadow are obtained by averaging the coordinates of all pixels within the shadow pixel region according to their positions. Based on the set of shadow contour sampling points, the contour shape similarity between the source image and the image to be tested is calculated. The contour shape similarity is measured by the shape context distance. The contour shape similarity is inversely proportional to the shape context distance. If the contour shape similarity is greater than the preset shape similarity threshold, then a one-to-one correspondence is performed according to the relative arc length along the contour, and the shadow centroid coordinates are directly paired to obtain shadow key point pairs. If the contour shape similarity is not greater than the preset shape similarity threshold, then shadow key point pairs cannot be established. The projected shadow features in the image to be tested are labeled as abnormal shadow masks. The steps of establishing kinematic correlation and predicting shadow features are skipped, and the process directly proceeds to the step of masking the global texture features based on the abnormal shadow mask and determining the background tampering test result.

5. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 4, characterized in that, The process of directly pairing the shadow centroid coordinates according to the relative arc length order along the contour to obtain shadow keypoint pairs includes: For each pair of shadow key points, a sparse displacement vector is obtained by subtracting the corresponding key point pixel coordinates in the source image from the key point pixel coordinates in the image to be tested. All sparse displacement vectors are arranged according to their source image coordinate positions to form a sparse displacement field. Two-dimensional thin-plate spline interpolation is used to extend the sparse displacement field to the unsampled pixel positions within the shadow pixel region, resulting in a dense displacement field covering all pixels. The theoretical spatial migration trajectory is obtained by arranging the fields along the space.

6. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 1, characterized in that, The step of determining the kinematic relationship between human posture features and projected shadow features between the source image and the image to be tested, based on the theoretical spatial migration trajectory and the displacement components of joints in human posture features, includes: The joint displacement vector is obtained by the difference between the two-dimensional coordinates of each joint point in the source image and the image to be tested. The directed line segment connecting adjacent joint points is denoted as the limb vector. The rotation component and scaling factor of the limb are obtained by taking the ratio of the angle between the limb vector of the source image and the limb vector of the test image. The translation component is obtained by taking the arithmetic mean of the joint displacement vector along the horizontal and vertical directions. The posture deformation descriptor is obtained by combining the translation component, the rotation component and the scaling factor. The shadow deformation descriptor is obtained by taking the overall translation, overall rotation angle and overall area change rate of the displacement vector of each pixel in the theoretical spatial migration trajectory. The linear mapping matrix from the attitude deformation descriptor to the shadow deformation descriptor is solved by least squares fitting. The coefficients of each row of the linear mapping matrix represent the contribution weight of each attitude deformation component to the corresponding shadow deformation component.

7. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 6, characterized in that, The step of performing a geometric transformation on the projected shadow features in the source image based on the kinematic correlation to obtain predicted shadow features corresponding to the human posture in the image to be tested includes: The pose deformation descriptor from the source image to the image under test is substituted into the linear mapping matrix in the kinematic correlation and right-multiplied to obtain the target shadow deformation descriptor containing the target translation component, the target rotation angle and the target area change rate. The target shadow deformation descriptor is then combined into a single geometric transformation matrix according to the synthesis rules of two-dimensional affine transformation. The coordinates of each pixel in the shadow pixel region of the source image are mapped according to the geometric transformation matrix to obtain the predicted shadow pixel scatter map.

8. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 7, characterized in that, The step of mapping the coordinates of each pixel within the shadow pixel region of the source image according to the geometric transformation matrix to obtain the predicted shadow pixel scatter map includes: For the unfilled pixel locations in the predicted shadow pixel scatter map that appear due to sparse mapping, a morphological closing operation is used to first expand along a preset structuring element and then erode along the same structuring element to obtain a predicted shadow mask that is internally continuous and has smooth boundaries. The predicted shadow mask is then used as the predicted shadow feature.

9. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 1, characterized in that, The process involves comparing the predicted shadow features with the projected shadow features in the image to be tested to obtain a feature difference value. If the feature difference value meets a preset condition, a valid shadow mask is obtained; if the feature difference value does not meet the preset condition, an abnormal shadow mask is obtained. This includes: Align the predicted shadow features and the projected shadow features in the same pixel coordinate system, take the intersection number and the union number of pixels for the pixel positions covered by the two shadow masks, subtract the intersection number from the union number of pixels and then divide by the union number of pixels to obtain the feature difference value that reflects the degree of spatial overlap between the two shadow masks; If the feature difference value does not exceed the preset deformation tolerance threshold, then the projected shadow feature is determined to conform to the kinematic correlation relationship, and the pixel position covered by the projected shadow feature is marked as a valid shadow mask; If the feature difference value exceeds the preset deformation tolerance threshold, the projected shadow feature is determined to be a lingering shadow detached from the human body posture, and the pixel position covered by the projected shadow feature is marked as an abnormal shadow mask.

10. The background tampering detection method based on human posture and shadow kinematic constraints according to claim 1, characterized in that, The process involves removing masked regions from global texture features based on legitimate and anomalous shadow masks, comparing the remaining pixel distributions at the same spatial locations between the source image and the image under test, and determining the background tampering detection result of the image under test, including: The pixel positions covered by at least one of the generated legal shadow masks and abnormal shadow masks are merged into a set of pixels to be removed. The pixel positions in the set of pixels to be removed are removed from the global texture features to obtain the remaining background pixel distribution of the source image and the remaining background pixel distribution of the image to be tested. The average difference of the remaining background is obtained by summing the absolute differences of the gray values ​​of the remaining background pixels at the same spatial location and then dividing by the total number of remaining pixel locations. If the average difference of the remaining background does not exceed a preset threshold, it is determined that the background has not been tampered with; otherwise, it is determined that the background has been tampered with.

Images