Image enhancement processing method, device, equipment and medium

Abstract

The application discloses an image enhancement processing method, device and equipment and a medium. The method comprises the following steps: obtaining an original training image of model training, and determining a human body region and an original background region based on the original training image; determining a human body skeleton connection line based on the human body region; determining a background expansion region based on the human body skeleton connection line; generating a synthetic background in the background expansion region at random, and fusing the synthetic background, the human body region and the original background region to generate an enhanced training image. The method dynamically generates a random synthetic background around the human body skeleton connection line, and can realize various background interference of the training image without relying on a real background image library, thereby improving the diversity and generalization of the training image and reducing the overfitting risk of the model during training by using the training image.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an image enhancement processing method, apparatus, device, and medium. Background Technology

[0002] With the large-scale implementation of technologies such as motion sensing interaction, motion capture, and human-computer collaboration, human pose estimation models are widely used in scenarios such as motion-sensing games, smart homes, and motion analysis. Among these technologies, to address the problem of background elements such as furniture, pedestrians, and changing lighting interfering with skeletal point detection in complex indoor scenes, a data augmentation method involving full-image background replacement is typically used when training the human pose estimation model. This involves first extracting the human body region by segmenting the model, then selecting scene images from a pre-collected library of real background images as the new background, and finally fusing the human body with the new background to generate training data.

[0003] However, full-image replacement methods based on real-world background image libraries are limited by the size of the library and cannot exhaustively cover all real-world scenes. The models have only seen a limited number of background types during training, and when deployed to new, unseen scenes, issues such as skeletal point drift and mislocalization of background objects still arise. Furthermore, the models are prone to learning spurious associations between the background and the human body, posing a risk of overfitting. Therefore, there is an urgent need for an image enhancement method that does not rely on a large-scale background image library and can dynamically generate diverse backgrounds. Summary of the Invention

[0004] This invention provides an image enhancement processing method, apparatus, device, and medium to address the problems of training images relying on large-scale background image libraries and insufficient background diversity in related technologies.

[0005] In a first aspect, the present invention provides an image enhancement processing method, characterized in that it includes: Obtain the original training images for model training, and determine the human body region and the original background region based on the original training images; Based on human body regions, determine the connections between human skeletons; Based on the lines connecting the human skeleton, the background expansion area is determined; Within the background augmentation area, a synthetic background is randomly generated, and the synthetic background, the human body region, and the original background region are merged to generate an enhanced training image.

[0006] In some embodiments, determining the human skeletal connections based on human body regions includes: Key point detection is performed on human body regions to obtain key points of the human skeleton; Based on the key points of the human skeleton, lines are connected according to the preset human skeleton topology to obtain the human skeleton connection.

[0007] In some embodiments, determining the background augmentation region based on human skeletal connections includes: Based on the lines connecting the human skeleton, determine the direction and width of the region expansion; Using the lines connecting the human skeleton as the axis, the background expansion area is determined based on the expansion direction and expansion width.

[0008] In some embodiments, determining the region expansion direction and region expansion width based on the human skeletal lines includes: The vertical and extended directions of the lines connecting the human skeleton are used as the directions for expanding the region. The expansion width of the region is determined based on the length of the connecting lines of the human skeleton and the corresponding key points of the human skeleton.

[0009] In some embodiments, within the background augmentation area, a synthetic background is randomly generated, including: The background expansion area is randomly divided to obtain multiple background sub-regions of different shapes; Fill the background sub-region with color and / or texture to obtain the composite background.

[0010] In some embodiments, color filling of the background sub-region includes: Obtain the human body color features of human body regions; The background color features are determined based on human body color characteristics and a preset color variation range; Based on the background color characteristics, the background sub-regions are filled with color.

[0011] In some embodiments, the synthetic background, the human body region, and the original background region are fused to generate an enhanced training image, including: The area in the original background region that overlaps with the composite background is replaced with the corresponding composite background to obtain the enhanced background region; The enhanced background region is stitched together with the human body region to obtain the enhanced training image.

[0012] In a second aspect, the present invention provides an image enhancement processing apparatus, comprising: The image segmentation module is used to obtain the original training images for model training and, based on the original training images, determine the human body region and the original background region. The connection determination module is used to determine the connections of the human skeleton based on human body regions; The expanded region determination module is used to determine the background expanded region based on the human skeleton connection lines; The image enhancement module is used to randomly generate a synthetic background within the background expansion area, and then merge the synthetic background, the human body region, and the original background region to generate an enhanced training image.

[0013] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described image enhancement processing method.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image enhancement processing method.

[0015] The aforementioned image enhancement processing method, apparatus, device, and medium achieve the following: First, acquire the original training images used for model training. Based on these images, determine the human body region and the original background region. Then, based on the human body region, determine the human skeleton connections. Next, based on the human skeleton connections, determine the background expansion region. Within the background expansion region, randomly generate a synthetic background. Finally, merge the synthetic background, the human body region, and the original background region to generate an enhanced training image. This method dynamically generates random synthetic backgrounds around the human skeleton connections, enabling diverse background interference in the training images without relying on a real background image library. This improves the diversity and generalization of the training images while reducing the risk of overfitting when using training images to train the model. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of an image enhancement processing method according to an embodiment of the present invention; Figure 2 This is another flowchart of an image enhancement processing method according to an embodiment of the present invention; Figure 3 This is another flowchart of an image enhancement processing method according to an embodiment of the present invention; Figure 4 This is another flowchart of an image enhancement processing method according to an embodiment of the present invention; Figure 5 This is another flowchart of an image enhancement processing method according to an embodiment of the present invention; Figure 6 This is a schematic block diagram of an image enhancement processing device according to an embodiment of the present invention; Figure 7 This is a schematic block diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0018] As an example, such as Figure 1 As shown, an image enhancement processing method is provided, including the following steps: S101, Obtain the original training images for model training, and determine the human body region and the original background region based on the original training images; S102, Based on the human body region, determine the connection lines of the human skeleton; S103, Determine the background expansion area based on the human skeleton connection lines; S104: Within the background expansion area, a synthetic background is randomly generated, and the synthetic background, the human body region, and the original background region are merged to generate an enhanced training image.

[0019] As an example, in step S101, the original training image for training the human pose estimation model can be obtained, and the original training image can be divided into regions, which are human regions representing the human body to be detected and original background regions representing scene content, so as to provide a location basis for subsequent regionalization enhancement.

[0020] In one embodiment, the image segmentation process can employ any segmentation algorithm that can distinguish between the human body and the background. The segmentation process can be completed offline and the segmentation mask can be stored before model training, or it can be calculated online in real time during training.

[0021] In one embodiment, after the region is divided, basic morphological processing can be performed on the segmentation results to smooth the edge burrs of the human body region, fill small area holes in the segmentation results, and avoid the jaggedness or gaps in the segmentation results from affecting the subsequent fusion effect.

[0022] For example, the original training images can be real-time user motion images captured by a motion-sensing camera, or labeled images from a publicly available human pose dataset. As long as the image contains human targets with corresponding skeletal keypoint annotations, it can be used for the enhancement processing of this method.

[0023] As an example, in step S102, the skeletal structure information of the human body can be extracted based on the segmented human body region, and the position lines of each segment of the human body skeleton in the 2D image plane can be determined, providing a precise benchmark for the subsequent localization of interference areas.

[0024] In other words, this step does not require the identification of the 3D skeletal structure of the human body, nor does it require depth information collected by a depth sensor. It only requires determining the line segments representing the direction of the human skeleton on a two-dimensional image, which can be adapted to the usage scenarios of ordinary RGB cameras.

[0025] In one embodiment, key joint positions of the human body can be identified by joint detection, and then the corresponding joints can be connected according to the human physiological structure to obtain skeletal lines. The number of key points and connection rules can be flexibly adjusted according to the annotation standards of the training dataset and the accuracy requirements of the application scenario.

[0026] In one embodiment, a preset label file corresponding to the original training can be read synchronously. This preset label file stores information such as the coordinates of human key points and the connection lines of human skeletons. By reading the label file, the positions of human key points and the connection lines of human skeletons can be obtained directly, thereby avoiding errors in online key point detection and reducing the amount of online computation.

[0027] For example, for posture models in everyday interactive scenarios such as family motion-sensing games, it is sufficient to identify only six main joints—shoulder, elbow, wrist, hip, knee, and ankle—and connect them to obtain the main skeletal lines of the limbs and torso to meet the enhancement needs of everyday interactive scenarios, without the need to additionally identify key points of delicate parts such as fingers.

[0028] As an example, in step S103, the target area for background enhancement, i.e. the background expansion area, can be delineated around the bones based on the determined human skeleton lines, so as to achieve precise enhancement of the easily disturbed areas around the bones and avoid the image distortion problem caused by full-image enhancement.

[0029] In other words, this step does not require a global replacement of the background of the entire image, but only enhances the area around the skeleton. This ensures the interference effect while preserving the true data distribution of the original training image as much as possible, avoiding the large difference between the enhanced image and the real scene, which could cause the model to learn false features.

[0030] In one embodiment, the range of the background expansion area can be adaptively adjusted according to the position, length and body part of the bone. Different area sizes and shapes are set for bones in different positions to achieve dynamic allocation of interference intensity and reduce unnecessary calculations while ensuring interference effect.

[0031] In one embodiment, the background expansion area can cover a small number of pixels around the human body edge, ensuring that the background around the human body contour is fully included in the enhancement range, avoiding leaving unenhanced areas at the junction of the human body and the background, so that the model can still rely on the original background information of the human body edge for prediction.

[0032] For example, for joints such as the wrist and ankle that are far from the camera and prone to prediction drift, the expansion area of ​​the corresponding bones can be appropriately expanded to improve the anti-interference ability of these error-prone parts; for parts with high prediction stability such as the torso, the expansion area can be appropriately reduced to reduce unnecessary computational overhead.

[0033] As an example, in step S104, a random synthetic background can be dynamically generated within the defined background expansion area, and then the synthetic background is fused with the human body area and the original background area of ​​the original training image according to the region mask to obtain the final enhanced training image that can be directly used for model training.

[0034] In other words, the composite background is generated randomly in real time during the enhancement process, rather than being selected from a pre-stored library of real background images. This allows for the generation of an infinite number of different background combinations, thus solving the problem of insufficient diversity in the background library and its inability to cover all real-world scenes.

[0035] In one embodiment, the synthetic background can be generated by randomly combining visually disruptive elements such as random color blocks and random textures. The shape, color, and density of the elements can be randomly varied to ensure that the synthetic background does not have a fixed semantic association with the real scene and cannot form a stable false correspondence with the human body.

[0036] In one embodiment, the fusion process is based on the segmented region mask to ensure that the effective pixels of the human body region are not modified, and only the background pixels in the expanded region are replaced. After fusion, the edges of the regions can be smoothed to eliminate splicing marks and ensure the visual naturalness of the enhanced image.

[0037] For example, the composite background can be a random patchwork of different colors, or a random texture such as stripes, grids, or noise. It can also be mixed with interference color blocks that are similar to human body color. As long as it can effectively interfere with the model and break the model's dependence on background information, it is fine. There is no need to deliberately simulate the specific content of the real scene.

[0038] In one optional embodiment, enhanced training images can be used to train human pose estimation models, human recognition models, etc., so that the models cannot rely on background information for prediction during the training process, thereby forcing the models to focus more on the features of the human body itself and improving the model's generalization ability and robustness in complex backgrounds.

[0039] In summary, this disclosure proposes an image enhancement processing method, comprising: acquiring the original training image for model training, and determining the human body region and the original background region based on the original training image; determining the human skeleton connection lines based on the human body region; determining the background expansion region based on the human skeleton connection lines; randomly generating a synthetic background within the background expansion region, and fusing the synthetic background, the human body region, and the original background region to generate an enhanced training image. This method dynamically generates a random synthetic background around the human skeleton connection lines, enabling diverse background interference in the training image without relying on a real background image library, thus improving the diversity and generalization of the training image, while reducing the risk of overfitting when using the training image to train the model.

[0040] In one embodiment, such as Figure 2 As shown, step S102, which is to determine the human skeleton connection lines based on the human body region, includes: S201, perform key point detection on the human body region to obtain key points of the human skeleton; S202, Based on the key points of the human skeleton, connect them according to the preset human skeleton topology to obtain the human skeleton connection.

[0041] As an example, in step S201, key point detection can be performed only on the human body region divided in step S101, without performing full-image detection on the entire original training image, thus reducing unnecessary computation and finally obtaining the two-dimensional pixel coordinates of the skeletal key points of the human body region.

[0042] In other words, the detection range of this step is limited to the area where the human body is located, and invalid detection will not be performed on the background area. This can improve the processing speed of key point detection and also prevent objects in the background that resemble human body structures (such as furniture edges or strip decorations) from being misidentified as human body key points.

[0043] In one embodiment, key point detection can select the set of key points to be detected according to the application scenario. It is not necessary to detect all human key points. The number and position of key points to be detected can be flexibly adjusted according to the accuracy requirements of the scenario, so as to strike a balance between accuracy and speed.

[0044] In one embodiment, each detected key point can have its confidence score determined simultaneously. For key points with a confidence score lower than a preset confidence threshold, they are identified as low-confidence key points. Linear interpolation can be performed at the positions of adjacent high-confidence key points with confidence scores higher than the preset confidence threshold to complete the connection and avoid the interruption or misalignment of subsequent connections due to missing key points.

[0045] For example, for lightweight interactive scenarios such as home motion-sensing games and fitness motion recognition, the system can focus on detecting 12 key points connecting the limbs and torso in 6 categories: shoulder, elbow, wrist, hip, knee, and ankle, while omitting key points such as the face and fingers that have less impact on the judgment of large movements, thereby further improving processing speed while meeting the accuracy requirements of motion recognition. For high-precision scenarios such as professional motion capture and rehabilitation training assessment, the system can add fine key points such as fingers, toes, and face to meet the needs of fine motion recognition.

[0046] As an example, in step S202, the detected human skeleton key points can be paired up and connected into straight line segments according to the pre-stored human skeleton connection rules, and finally a human skeleton connection line covering the main human skeleton can be obtained.

[0047] In other words, the preset human skeleton topology is a predefined fixed key point pairing rule. It does not require recalculating the connection relationship every time it is enhanced. It only needs to match the corresponding key points according to the preset rule, which ensures that the skeleton connection logic is consistent in different images and avoids the problem of inconsistent connection methods for the same bone segment in different images.

[0048] In one embodiment, the position of the key points can be checked for rationality before connecting the lines. If the distance or relative position of the two key points to be connected does not conform to the normal range of human physiological structure (for example, the distance between the two key points exceeds the limit of human limb length, or the wrist key point appears below the hip joint, etc.), it can be determined that the key point detection is wrong. The connection of the bone segment is skipped or the position is corrected by the adjacent key points before connecting the lines, so as to avoid the subsequent misalignment of the enhanced area due to incorrect connection.

[0049] In one embodiment, after the lines are connected, the generated line segments can be filtered for validity, eliminating redundant lines that are too short or erroneous lines that fall completely outside the human body area, and retaining only valid lines that fall within the human body area and whose length conforms to the physiological range of the human body, as the benchmark for subsequently determining the background expansion area.

[0050] For example, a preset human skeleton topology suitable for lightweight interaction scenarios can be defined as: left shoulder - left elbow, left elbow - left wrist, right shoulder - right elbow, right elbow - right wrist, left shoulder - right shoulder. By connecting the corresponding key points according to this rule, all the major bones of the human upper limb can be connected, which can meet the enhancement requirements of motion-sensing interaction scenarios.

[0051] In one embodiment, such as Figure 3 As shown, step S103, which is to determine the background expansion area based on the human skeleton connection lines, includes: S301, based on the human skeleton connection lines, determine the region expansion direction and region expansion width; S302, using the human skeleton as the axis, and based on the region expansion direction and region expansion width, determine the background expansion region.

[0052] As an example, in step S301, for each independent human skeletal connection, the expansion direction and expansion width corresponding to that skeletal segment can be determined respectively. The parameters corresponding to different human skeletal connections can be the same or different.

[0053] In other words, we can customize exclusive expansion rules for each human skeleton connection and then generate specific pixel regions according to the rules, avoiding problems such as image distortion and computational waste caused by indiscriminate expansion of the entire image.

[0054] In one embodiment, the direction of the expansion area can be flexibly adjusted according to the actual orientation of the bone, and is not limited to a fixed up, down, left, or right direction. The direction can be adaptively adjusted according to the tilt angle of the bone to ensure that the expansion area always surrounds the bone itself and that there is no problem of the area deviating from the bone.

[0055] In one embodiment, the region expansion width can be flexibly adjusted according to bone type, location, and model training stage. It is not necessary for all bones to use the same width value. A larger width can be set for bones that are prone to prediction errors, and a smaller width can be set for bones that are stable in prediction, so as to achieve dynamic allocation of interference intensity.

[0056] For example, in the early stages of training, the model has not yet mastered the basic human body structure, so a small global expansion width can be used to avoid strong interference causing the model to fail to converge; in the later stages of training, the model has already acquired basic key point prediction capabilities, so the global expansion width can be increased to further enhance the interference intensity.

[0057] As an example, in step S302, each human skeleton line can be used as the geometric center, and a closed pixel region can be generated as the expansion region of that skeleton according to the expansion direction and width determined in S301. After taking the union of the expansion regions of all skeletons, the final total background expansion region of the entire image is obtained.

[0058] In one optional embodiment, the vertical direction and extension direction of the human skeleton connection line can be used as the region expansion direction; the region expansion width is determined based on the length of the human skeleton connection line and the corresponding human skeleton key points.

[0059] The vertical direction is the two directions that form a 90-degree angle with the direction of the bone connection, used to cover the background on the left and right sides of the bone; the extension direction is the direction that extends outward from the key points at both ends along the direction of the bone connection, used to cover the background around the joints; the area expansion width is determined based on the length of the human skeleton connection and the corresponding human skeleton key points. Bones with longer lengths and bones corresponding to distal key points that are prone to drift, such as wrists / ankles, are given a larger expansion width, while bones with shorter lengths and bones that are predicted to be stable, such as the torso, are given a smaller width.

[0060] Furthermore, circular expansions can be made at both ends of the lines connecting the human skeleton (i.e., key points of the human skeleton) to fully cover the background on both sides of the skeleton, as well as the easily misjudged areas around the joints.

[0061] In one embodiment, an elliptical expansion rule can be used as an alternative: the midpoint of each bone segment connection is taken as the center of the ellipse, the length of the bone connection is taken as the major axis of the ellipse, and the set expansion width is taken as the minor axis of the ellipse to generate an elliptical expansion region; the region generated in this way has smooth edges without sharp corners, the edge processing during subsequent fusion is simpler, and the generated enhanced image is more natural.

[0062] In one embodiment, a Gaussian gradient expansion rule can be used as an alternative implementation: with the skeleton connection line as the center, different expansion probabilities are set outward according to a two-dimensional Gaussian distribution. The closer the pixel is to the skeleton, the higher the probability of it being included in the expansion area, and the farther the pixel is from the skeleton, the lower the probability of it being included. In the end, an expansion area with a soft edge gradient is generated, so that the generated area has no hard boundary and no obvious splicing marks appear after fusion.

[0063] In one embodiment, the region expansion width can also be adaptively adjusted in conjunction with the resolution of the original training image: in high-resolution images, the proportion of human body pixels is larger, so the expansion width can be increased accordingly; in low-resolution images, the proportion of human body pixels is smaller, so the expansion width can be decreased accordingly, ensuring that the proportion of the expanded region relative to the human body remains stable and that the interference intensity does not become inconsistent due to changes in image resolution.

[0064] In one embodiment, such as Figure 4 As shown, step S104, which involves randomly generating a composite background within the background expansion area, includes: S401, randomly divide the background expansion area to obtain multiple background sub-regions of different shapes; S402, fill the background sub-region with color and / or texture to obtain a composite background.

[0065] As an example, in step S401, the background expansion area determined in step S103 can be randomly divided into multiple independent small areas that do not overlap and have random shapes, i.e., background sub-regions, in preparation for subsequent block filling.

[0066] In other words, the continuous background expansion area can be randomly divided into several small blocks, each of which can be independently filled with different content. Finally, these blocks are spliced ​​together to form a composite background with strong interference, avoiding the problem of insufficient interference intensity and insufficient background diversity caused by filling the entire area with a single color or texture.

[0067] In one embodiment, a random polygon segmentation method can be used to divide the background sub-regions. That is, several uniformly distributed seed points are randomly generated within the expanded region, and Voronoi polygons are generated based on the seed points to obtain multiple irregular polygon sub-regions with random numbers of sides, sizes, and angles. The sub-regions generated in this way have natural and irregular shapes and are more disruptive.

[0068] In one embodiment, a random rectangle segmentation method can be used to divide the background sub-region: several rectangles of different sizes and aspect ratios are randomly generated within the expanded area. The rectangles can be randomly rotated within the range of 0-90 degrees, and the edge areas not covered by the rectangles are automatically used as independent sub-regions.

[0069] In one embodiment, after the background sub-region is divided, region optimization can be performed automatically, merging excessively small background sub-regions with areas smaller than a preset threshold into adjacent background sub-regions with larger areas, so as to avoid the background sub-regions becoming too fragmented and becoming meaningless noise points, which would affect the overall interference effect.

[0070] As an example, in step S402, the filling method can be selected independently for each of the divided background sub-regions. You can choose to fill with color alone, fill with texture alone, or overlay color and texture filling at the same time. After all sub-regions are filled, they are naturally stitched together to form a complete composite background.

[0071] In other words, the filling method and content of each background sub-region are independent and random, and there will be no large areas of the same color or texture. This ensures the randomness and diversity of the synthesized background, making it impossible for the model to find a fixed pattern in the synthesized background. It can only focus on the structural features of the human body itself.

[0072] In one embodiment, the proportion of background sub-regions with different filling methods can be allocated according to a preset ratio. For example, 60% of the background sub-regions are filled with solid colors, 30% of the background sub-regions are filled with textures, and the remaining 10% of the background sub-regions are filled with a mixture of color and texture. This ensures the richness of the synthesized background elements while controlling the amount of computation within a reasonable range.

[0073] In one embodiment, when filling with color, at least two colors are selected for filling multiple background sub-regions to avoid insufficient interference caused by single-color filling. Specifically, each background sub-region randomly selects a color from a preset color library, which contains multiple combinations of hue, saturation, and brightness. In one embodiment, the filling process can automatically perform adjacent area verification: when performing color filling, the color difference between adjacent sub-regions must be greater than a preset color difference threshold to avoid adjacent sub-regions being too similar in color to be distinguishable; when performing texture filling, adjacent sub-regions must select different types of textures to avoid adjacent regions having similar textures that cause them to appear as one piece, thus losing the effect of segmentation interference.

[0074] In one embodiment, the filling priority can be set based on the distance between the sub-region and the bone connection line: for the near terminal region within 10 pixels of the bone connection line, high saturation color and high contrast texture are preferred for filling to increase the interference intensity around the bone; for the far terminal region far from the bone connection line, low saturation color and weak contrast texture can be selected for filling to ensure the interference effect while avoiding the synthesized background from being too abrupt.

[0075] For example, the available texture types include, but are not limited to, horizontal stripes, vertical stripes, diagonal stripes, square grids, Gaussian noise, checkerboard, linear gradients, radial gradients, and scattered textures. When filling, you can randomly select a single texture or choose to overlay two textures to further enhance the diversity of the composite background.

[0076] In one embodiment, after all sub-regions are filled, a small number of pixels (e.g., 1-2 pixels) of Gaussian blur can be applied to the stitching edges between sub-regions to make the transition between different sub-regions more natural, avoid hard edges that cause excessive artificial traces in the composite background, and affect the realism of the enhanced image.

[0077] In one optional embodiment, when filling the background sub-region with color, the human body color features of the human body region can be obtained; based on the human body color features and a preset color change range, the background color features are determined; and based on the background color features, the background sub-region is filled with color.

[0078] In other words, when filling in colors, a certain proportion of similar color interference blocks can be generated by referring to the human body's own color. This allows for the arrangement of visual elements with similar colors to the human body around the skeleton, further enhancing the interference intensity of the background and preventing the model from recognizing the human body solely based on color differences. Instead, the model learns more fundamental features of the human skeletal structure.

[0079] In one embodiment, when acquiring human body color features, the average RGB value and color distribution range of all pixels within the human body region can be statistically analyzed to determine the human body color features of the human body region.

[0080] In one embodiment, when determining the background color features, two generation logics can be allocated according to a preset ratio: some background sub-regions randomly select colors within a preset range of human body color features to generate interference color blocks similar to human body colors; the remaining background sub-regions randomly select colors in the full RGB color space, and ensure that the color difference with adjacent background sub-regions is greater than a preset threshold, taking into account both the targeting of interference and the diversity of backgrounds.

[0081] In one embodiment, interference color blocks that are similar to human body color can be preferentially placed in the background sub-regions that are close to the lines connecting the human skeleton (e.g., within 5 pixels). Interference that is similar to human body color can be placed next to the skeleton to further increase the difficulty for the model to determine the position of the skeleton. Background sub-regions that are far from the skeleton can use random colors to avoid too many similar color blocks causing the overall synthesized background to be too similar to human body color.

[0082] In one embodiment, the preset color variation range can be dynamically adjusted according to the training stage: in the early stage of training, the variation range is set to be larger, and the color difference between similar color blocks and human body is slightly larger, so as to avoid excessive interference that causes the model to fail to converge; in the later stage of training, the variation range is set to be smaller, and the color difference between similar color blocks and human body is closer, gradually increasing the difficulty of interference and adapting to the learning progress of the model.

[0083] In one embodiment, such as Figure 5 As shown, step S104 involves fusing the synthetic background, the human body region, and the original background region to generate an enhanced training image, including: S501, Replace the area in the original background region that overlaps with the composite background with the corresponding composite background to obtain the enhanced background region; S502 stitches the enhanced background region with the human body region to obtain an enhanced training image.

[0084] As an example, in step S501, the positions of the original background area and the generated composite background can be compared pixel by pixel. The original background pixels at the overlapping positions of the two areas are replaced with the corresponding pixels of the composite background, while the original background pixels at the non-overlapping positions remain unchanged, and finally the enhanced background area with the replacement is obtained.

[0085] In other words, when determining the background enhancement region, only the original background region covered by the synthetic background is modified, while the original background region not covered by the synthetic background is completely preserved. The background of the entire image is not completely replaced. This achieves targeted interference around the skeleton while preserving the true distribution of the original scene to the greatest extent, avoiding the model learning false features due to the large difference between the enhanced image and the real scene.

[0086] In one embodiment, the pixel replacement process can traverse the image row by row and column by column. If the current pixel belongs to the original background area and falls within the coverage of the composite background, the pixel value is directly replaced with the pixel value of the corresponding position in the composite background. If the current pixel belongs to the original background area but is not within the range of the composite background, the original pixel value is retained.

[0087] In one embodiment, the pixel values ​​of the composite background can be slightly and randomly perturbed during the replacement process, such as randomly adjusting the pixel values ​​within a brightness range of ±5, to avoid the pixels of the composite background being too regular and the artificial traces being too heavy, thereby further enhancing the realism of the composite background.

[0088] In one embodiment, if the synthesized background has small holes or gaps due to issues such as region division and edge calculation, it can be automatically filled by interpolation using surrounding synthesized background pixels to avoid missing replacement of original background pixels, ensuring that the background around the skeleton is completely replaced without leaving any interference blind spots.

[0089] For example, if in a motion-sensing image, the synthesized background only covers a 100*100 pixel area around the wrist, then only the background pixels in that area will be replaced with randomly synthesized color blocks and textures. Other background elements in the image, such as the sofa, walls, and furniture in the living room, will be completely preserved and will not be modified.

[0090] As an example, in step S502, the replaced enhanced background area and the previously determined human body area can be stitched together according to pixel position, while the pixel values ​​of the human body area remain unchanged. After stitching, a complete enhanced training image that can be used for training is obtained.

[0091] In other words, the core pixels of the human body region will not be modified during the stitching process. Even if the synthesized background covers the edge of the human body, the effective pixels of the human body itself will be completely preserved, ensuring a strict correspondence between the human body pixels and the key point annotations, and preventing the introduction of training label noise due to the modification of human body pixels.

[0092] In one embodiment, the stitching process is based on the position mask of the human body region. The pixel positions marked as human body in the mask directly use the original human body pixels, while the pixel positions marked as background in the mask use the pixels of the enhanced background region.

[0093] In one embodiment, after the stitching is completed, the boundary edge (e.g., 1-3 pixels) between the human body area and the enhanced background area can be Gaussian blurred to make the transition between the human body and the synthetic background more natural, eliminate obvious edges generated by hard stitching, and avoid the model learning false features of the stitching edge.

[0094] In an optional embodiment, the boundary of the human body region can also be replaced by enhancing the background region. For example, the edge pixels within the outermost 3-pixel range of the human body region can be replaced with synthetic background pixels with a low probability of 10%-20% to simulate the situation where the human body edge is slightly occluded by small objects in a real scene, further improving the robustness of the model to edge occlusion. This replacement only targets the outermost edge pixels and will not modify the pixels in the core area of ​​the human body, thus not introducing labeling errors.

[0095] In one embodiment, an image enhancement processing apparatus is provided, which corresponds one-to-one with the image enhancement processing methods described in the above embodiments. For example... Figure 6 As shown, the image enhancement processing device includes an image segmentation module 601, a connection determination module 602, an expansion region determination module 603, and an image enhancement module 604. Detailed descriptions of each functional module are as follows: The image segmentation module 601 is used to acquire the original training images for model training and, based on the original training images, determine the human body region and the original background region. The connection determination module 602 is used to determine the connection lines of the human skeleton based on the human body region; The extended region determination module 603 is used to determine the background extended region based on the human skeleton connection lines; Image enhancement module 604 is used to randomly generate a synthetic background within the background expansion area and merge the synthetic background, the human body area and the original background area to generate an enhanced training image.

[0096] In one embodiment, the connection determination module 602 is further configured to perform key point detection on the human body region to obtain key points of the human skeleton; Based on the key points of the human skeleton, lines are connected according to the preset human skeleton topology to obtain the human skeleton connection.

[0097] In one embodiment, the expansion area determination module 603 is further configured to determine the expansion direction and expansion width of the area based on the human skeleton connection lines. Using the lines connecting the human skeleton as the axis, the background expansion area is determined based on the expansion direction and expansion width.

[0098] In one embodiment, the expansion region determination module 603 is further configured to use the vertical direction and the extension direction of the human skeleton connection line as the region expansion direction. The expansion width of the region is determined based on the length of the connecting lines of the human skeleton and the corresponding key points of the human skeleton.

[0099] In one embodiment, the image enhancement module 604 is further configured to randomly divide the background expansion area to obtain multiple background sub-regions of different shapes; Fill the background sub-region with color and / or texture to obtain the composite background.

[0100] In one embodiment, the image enhancement module 604 is further configured to acquire human body color features of the human body region; The background color features are determined based on human body color characteristics and a preset color variation range; Based on the background color characteristics, the background sub-regions are filled with color.

[0101] In one embodiment, the image enhancement module 604 is further configured to replace the region in the original background region that overlaps with the synthetic background with the corresponding synthetic background to obtain an enhanced background region; The enhanced background region is stitched together with the human body region to obtain the enhanced training image.

[0102] This invention provides an image enhancement processing apparatus, comprising: an image segmentation module for acquiring original training images for model training and determining human body regions and original background regions based on the original training images; a connection determination module for determining human skeleton connections based on the human body regions; an expansion region determination module for determining background expansion regions based on human skeleton connections; and an image enhancement module for randomly generating a synthetic background within the background expansion regions and fusing the synthetic background, human body regions, and original background regions to generate an enhanced training image. This apparatus dynamically generates random synthetic backgrounds around the human skeleton connections, enabling diverse background interference in training images without relying on a real background image library. This improves the diversity and generalization of training images while reducing the risk of overfitting when using training images to train the model.

[0103] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database contains data used in the image enhancement processing method. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program can implement an image enhancement processing method.

[0104] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an image enhancement processing method.

[0105] In one embodiment, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements an image enhancement processing method.

[0106] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), IAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0108] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An image enhancement processing method characterized by, include: Obtain the original training images for model training, and determine the human body region and the original background region based on the original training images; Based on the aforementioned human body region, the lines connecting the human skeleton are determined; Based on the aforementioned human skeletal lines, the background expansion area is determined; Within the background expansion area, a synthetic background is randomly generated, and the synthetic background, the human body area, and the original background area are merged to generate an enhanced training image.

2. The method according to claim 1, characterized in that, Determining the human skeletal connections based on the human body region includes: Key point detection is performed on the human body region to obtain key points of the human skeleton; Based on the key points of the human skeleton, lines are connected according to a preset human skeleton topology to obtain the human skeleton connection.

3. The method according to claim 1, characterized in that, The step of determining the background expansion area based on the human skeletal connections includes: Based on the lines connecting the human skeleton, the direction and width of the region expansion are determined; Using the line connecting the human skeleton as the axis, the background expansion area is determined based on the expansion direction and the expansion width of the region.

4. The method according to claim 3, characterized in that, The determination of the region expansion direction and region expansion width based on the human skeletal connections includes: The vertical direction and the extension direction of the line connecting the human skeleton are used as the expansion direction of the region. The expansion width of the region is determined based on the length of the connecting lines of the human skeleton and the corresponding key points of the human skeleton.

5. The method according to claim 1, characterized in that, The step of randomly generating a synthetic background within the background expansion area includes: The background expansion area is randomly divided to obtain multiple background sub-regions of different shapes; The background sub-region is filled with color and / or texture to obtain the composite background.

6. The method according to claim 5, characterized in that, The step of filling the background sub-region with color includes: Obtain the human body color features of the human body region; Based on the human body color characteristics and the preset color variation range, the background color characteristics are determined; Based on the background color features, the background sub-region is filled with color.

7. The method according to claim 1, characterized in that, The step of fusing the synthetic background, the human body region, and the original background region to generate an enhanced training image includes: The region in the original background region that overlaps with the synthesized background is replaced with the corresponding synthesized background to obtain the enhanced background region; The enhanced background region is stitched together with the human body region to obtain the enhanced training image.

8. An image enhancement processing apparatus, characterized in that, include: The image segmentation module is used to acquire the original training images for model training, and to determine the human body region and the original background region based on the original training images. The connection determination module is used to determine the connection lines of the human skeleton based on the human body region; An expanded area determination module is used to determine the background expanded area based on the human skeleton connection lines; An image enhancement module is used to randomly generate a synthetic background within the background expansion area, and to fuse the synthetic background, the human body area, and the original background area to generate an enhanced training image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image enhancement processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the image enhancement processing method as described in any one of claims 1 to 7.

Images