A method and system for product image segmentation in automatic photography equipment
By employing a dual-resolution processing architecture and multi-evidence gating constraints for depth estimation, combined with texture statistics and depth estimation, the challenge of product image segmentation in white-on-white scenarios is solved, realizing a high-precision and stable product image segmentation method and system suitable for automatic photography equipment.
Patent Information
- Application Number
- CN202611349471.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-02
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to effectively separate white products from white backgrounds in automated photography equipment, leading to missing product components and incorrect hole handling in white-on-white scenes. They also fail to balance edge quality and memory usage in high-resolution images and cannot be deployed locally.
A dual-resolution processing architecture is adopted, combining texture statistics and depth estimation. A soft foreground alpha image is generated through a matting neural network. Hole tri-state discrimination is performed by combining coarse-quantized gray-level co-occurrence matrix texture statistics and double background white difference. Closed holes are filled, preserved and manually verified. The outer contour is corrected by monocular depth estimation with multi-evidence gating constraints. Edge refinement and hole repair are performed to generate RGBA segmentation image.
It achieves accurate segmentation of product images in white-on-white scenes, repairs the through-hole caking defects caused by neural networks, avoids accidental modifications, meets the controllable operation requirements of production lines, can process ultra-high resolution images on consumer-grade hardware, eliminates synthetic white edges, and supports localized deployment.
Smart Images

Figure CN123118685A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for product image segmentation for automatic photography equipment. Background Technology
[0002] The e-commerce industry extensively uses automated photography equipment for batch product photography. Products are placed on a white surface with a white or light gray gradient background to capture high-resolution original images. These images then need to be automatically segmented to obtain foreground images. This scenario presents technical challenges related to white-on-white technology. The white product and white background are similar in color, leading to the loss of color, texture, and edge signals. Simultaneously, it's necessary to distinguish three types of similar-looking areas: realistic cutouts need to retain transparency, fabric texture holes need to be filled, and undetected internal areas of the product need to be restored. Furthermore, it's crucial to balance edge quality with memory usage in high-resolution image processing.
[0003] Existing cloud-based commercial image matting services sometimes result in the complete loss of product components, delivering erroneous results directly without distinguishing the processing logic for different holes, posing data security risks and making local deployment impossible. General prompt-based segmentation models are limited by input resolution, easily losing small white structures, and misidentifying display stands as foreground elements. Their parameters are difficult to generalize across different product styles, and relying on color processing in white-on-white scenes directly damages the white product subject.
[0004] Using end-to-end high-resolution matting models alone can easily treat real holes as solid structures, leading to missed detections of light-colored cavities and soles, white edges in the output, and a lack of semantic discrimination capabilities for holes, making it unsuitable for manual review in production environments. Traditional chroma keying and background subtraction methods are limited by contact shadows and white-on-white conditions, lacking stable separable thresholds and resulting in poor segmentation performance. Existing solutions cannot simultaneously address the multiple practical requirements of semantic hole discrimination, controllable processing of non-signal areas, balanced computing power and memory at high resolutions, and local deployment with zero marginal cost. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides a method and system for product image segmentation in automatic photography equipment. The technical solution is as follows: A method for product image segmentation for automatic photography equipment includes the following steps: Step 1: Acquire product images captured by an automatic photography device. The photography device has a background component with a first white background and a product support platform with a second white background. Retain a full-resolution copy of the image and generate a working resolution copy. Step 2: Input the working resolution copy into the trained matting neural network to generate a soft foreground Alpha image. After binarization and connected component cleanup, the basic contour is obtained, and all closed holes corresponding to the basic contour are extracted. Step 3: For each closed hole, combine the coarse gray-level co-occurrence matrix texture statistics inside the hole and the color difference between the hole color and the first white background and the second white background, respectively, and output a three-state decision: fill, retain, and manually check mark. Step 4: Based on the three-state decision processing, close the holes and perform hole repair processing on the real through holes on the white surface that are glued by the model. Step 5: The monocular depth estimation constrained by multi-evidence gating is used to perform correction only on the outer contour, and the depth estimation does not change the three-state decision result of the closed hole; Step 6: Perform edge refinement in the narrow contour region of the full-resolution image to generate a full-resolution RGBA segmentation image, and simultaneously output manually verified labels carrying structured information.
[0006] Optionally, the coarse quantization of the gray-level co-occurrence matrix texture statistics in step 3 is obtained by quantizing the image L* channel to 4-16 levels, calculating the local gray-level co-occurrence matrix contrast for multi-directional offsets and normalizing it; the closed hole processing also includes the fine slit dual signal criterion, background color gate judgment, and exception judgment of the joint between the platform and the background arc; the fine slit dual signal criterion is: the filling operation is only performed when the texture of the hole erosion area is greater than the texture threshold, and the difference between the hole color and the two background whites is greater than the color difference threshold.
[0007] Optionally, the first background white and the second background white can be automatically calibrated by the median pixel value of the corner pixels of the image and the median pixel value of the horizontal band below the product outline, or can be pre-calibrated by the camera device at the factory.
[0008] Optionally, step 4, the hole repair process, includes a closed-loop hole opening mechanism, a confidence wall hole opening mechanism, and a reserved hole trimming mechanism. Closed-loop aperture mechanism: Candidate regions that are surrounded by high-confidence solid foregrounds, have low alpha, flat interiors, and areas reaching a threshold are forcibly set to transparent; Confidence wall opening mechanism: Using a verified transparent core as a seed, real through holes are obtained by growing outward from the surrounding high-confidence product wall; Preservation of hole trimming mechanism: Using the color of the corroded area inside the hole as a local background reference, the background debris attached to the hole edge is removed by growing outward under multiple conditions.
[0009] Optionally, step 5, monocular depth estimation correction, also includes image-by-image self-calibration of depth bands, including the following steps: Catastrophic missing filling, simultaneously satisfying multiple conditions such as convex hull deficiency, hesitant pixel ratio, severity area gate, and being within the depth band to recover lost foreground regions; Background clipping trimming: When all three pieces of evidence—depth location, background color, and background depth level—are met, the mistakenly included background area is trimmed. The veto is reserved; a reserved hole will only be filled if both conditions are met simultaneously. Depth estimation does not overturn the three-state decision of the hole throughout the entire process.
[0010] Optionally, step 6 edge refinement uses the precise mode: after upsampling the image, a three-part map is constructed, and the three-part map matting network is run along the contour to solve the unknown region Alpha; a topology protection region is set to force the foreground to remain solid and original in color within the protection region, the edge of the hole retains the anti-aliasing effect, and foreground desmearing and white edge elimination are only performed on the narrow edge band.
[0011] Optionally, a switchable fast mode is also included, which uses guided filtering to fit the edges of the full-resolution image, combined with hierarchical curve mapping to eliminate the low alpha tail on the background side, thereby eliminating white edges and improving processing speed.
[0012] Optionally, the longest side of the working resolution copy is set to 1536-4096 pixels. All semantic and topological discrimination operations are performed on the working resolution copy, while only the edge thinning operation is performed within the narrow contour band of the full-resolution image.
[0013] Optionally, all modifying decisions require at least two independent pieces of evidence; if the conditions are not met, the corresponding changes will not be implemented. The manual verification markers carry hole type, area ratio, texture value, color difference, and bounding box structure information.
[0014] A product image segmentation system for an automatic photography device, used to implement a product image segmentation method for an automatic photography device, the system comprising: The dual-resolution loading module is used to acquire product images, save a full-resolution copy, and generate a working resolution copy. The soft mask generation module is used to call the matting neural network to generate a soft foreground alpha image and output a set of basic contours and closed holes. The hole tri-state discrimination module is used to combine coarse-quantized gray-level co-occurrence matrix texture statistics and double-background white difference to output a tri-state decision of filling, retaining, and manually verifying the marking of closed holes; The white hole repair module is used to perform through hole repair for closed loop opening, confidence wall opening, and retaining hole trimming; The depth-assisted outer contour correction module is used to run a monocular depth estimation network to correct the outer contour under multi-evidence gating without altering the three-state decision of the hole. The full-resolution edge refinement module is used to perform edge solving, topology protection, and foreground desmearing in narrow contour bands, and outputs an RGBA image. The review mark output module is used to output manual review marks carrying structured information.
[0015] In summary, the present invention has at least one of the following beneficial technical effects: This invention provides a method and system for product image segmentation using automatic photography equipment. It utilizes two types of background white provided by the photography equipment as prior information and combines texture statistics to achieve three-state discrimination of closed holes. It can perform differentiated processing on real hollows, fabric texture holes, and missed product interiors, effectively solving the problem of white-on-white scene segmentation.
[0016] Three types of white-on-white hole remediation mechanisms are implemented to correct the real hole-filling defects caused by the neural network. The monocular depth estimation uses multi-layered evidence gating, which is only used for outer contour correction and will not corrupt the hole detection results. This approach can both recover catastrophic missed detection areas and avoid the side effects of erroneous modifications.
[0017] For areas lacking valid pixel signals, manual verification markers are output to avoid silently outputting erroneous segmentation results, meeting the controllable operation requirements of the production line. A dual-resolution processing architecture is adopted, with semantic topology operations completed at the working resolution and edge refinement performed only in the narrow contour band of the full-resolution model. This allows for the processing of ultra-high-resolution images on consumer-grade hardware, eliminating composite white edges and controlling memory peaks.
[0018] The entire solution can be deployed entirely locally, without the need to transmit image data externally. There is no additional marginal cost for processing a single image. Global parameters can be adapted to multiple product styles without the need for individual parameter tuning. It has high segmentation accuracy and stable and reliable operation. Attached Figure Description
[0019] Figure 1 This is a flowchart illustrating a product image segmentation method for an automatic photography device according to the present invention. Figure 2 This is a flowchart of the closed hole three-state discrimination decision-making process in a specific embodiment of the present invention; Figure 3 This is a schematic diagram of the three mechanisms for remedying white holes in a specific embodiment of the present invention; Figure 4 These are schematic diagrams illustrating multiple sets of footwear product test samples in specific implementation examples of this invention. Detailed Implementation
[0020] The present invention will be further described in detail below with reference to the accompanying drawings.
[0021] This invention discloses a method and system for product image segmentation in automatic photography equipment.
[0022] Reference Figures 1-4 Example 1: A method for segmenting product images using an automatic photography device, comprising the following steps: Step 1: Acquire product images captured by an automatic photography device. The photography device has a background component with a first white background and a product support platform with a second white background. Retain a full-resolution copy of the image and generate a working resolution copy. Step 2: Input the working resolution copy into the trained matting neural network to generate a soft foreground Alpha image. After binarization and connected component cleanup, the basic contour is obtained, and all closed holes corresponding to the basic contour are extracted. Step 3: For each closed hole, combine the coarse gray-level co-occurrence matrix texture statistics inside the hole and the color difference between the hole color and the first white background and the second white background, respectively, and output a three-state decision: fill, retain, and manually check mark. Step 4: Based on the three-state decision processing, close the holes and perform hole repair processing on the real through holes on the white surface that are glued by the model. Step 5: The monocular depth estimation constrained by multi-evidence gating is used to perform correction only on the outer contour, and the depth estimation does not change the three-state decision result of the closed hole; Step 6: Perform edge refinement in the narrow contour region of the full-resolution image to generate a full-resolution RGBA segmentation image, and simultaneously output manually verified labels carrying structured information.
[0023] By adopting the above technical solution, in step 1, the product image captured by the automatic photography device is obtained. The photography device has a background component with a first white background and a product carrying platform with a second white background. The full-resolution copy of the image is retained and a working resolution copy is generated.
[0024] In this embodiment, the original image resolution is between 24 and 60 megapixels. The full-resolution copy completely preserves the pixel information of the original image and is used for final edge thinning and result output. The working resolution copy is used to handle all semantic judgments and topological operations, thereby controlling the overall computational load and avoiding memory overflow problems caused by ultra-high resolution images.
[0025] Step 2: Input the working resolution copy into the trained matting neural network to generate a soft foreground alpha image. After binarization and connected component cleanup, the basic contour is obtained, and all closed holes corresponding to the basic contour are extracted.
[0026] The image matting neural network possesses the ability to semantically separate the outlines of white-on-white scenes, and the output soft foreground alpha image records the foreground opacity for each pixel. After binarization, connected component cleanup is performed, preserving the main connected components and secondary connected components that meet the area ratio, ensuring that no parts are missing when multiple items are placed in pairs. The difference operation is performed between the basic outline and the image after filling holes to separate all closed holes inside the image.
[0027] Step 3: For each closed hole, combine the coarse gray-level co-occurrence matrix texture statistics inside the hole and the color difference between the hole color and the first and second background whites, respectively, and output a three-state decision: fill, retain, and manually verify the label.
[0028] Areas with holes may appear similar but require entirely different processing logic, making differentiation impossible using color information alone. Texture statistics can identify whether a hole belongs to the surface of the product or the background seen through the hole. Dual white backgrounds provide a color reference inherent to the scene, and the combination of both enables tri-state differentiation. For areas where the signal is insufficient for determination, the segmentation result is not automatically inferred but manually verified and marked before being output.
[0029] Step 4: Based on the three-state decision processing, close the holes and perform hole repair processing on the real through holes on the white surface that are glued by the model.
[0030] When both the product and the background are white, neural networks tend to mistake real, transparent holes for solid objects, forming semi-solid closed loops. This step is specifically designed to repair this type of failure scenario, restoring the transparency of the holes without damaging the fabric texture or the solid areas of the product.
[0031] Step 5: Monocular depth estimation with multi-evidence gating is used to correct only the outer contour, and the depth estimation does not change the three-state decision result of the closed hole.
[0032] Depth information can reflect the spatial difference between the product and the background, but the depth signal itself contains noise. This method limits the depth to the outer contour only, and all adjustments must meet multiple evidentiary conditions to avoid the depth signal from damaging holes that have already been determined, thus preventing incorrect filling or incorrect opening.
[0033] Step 6: Perform edge refinement in the narrow contour region of the full-resolution image to generate a full-resolution RGBA segmentation image, and simultaneously output manually verified labels carrying structured information.
[0034] Edge thinning operates only on a narrow band around the contour, avoiding the need to process the entire high-resolution image, thus balancing sub-pixel edge quality with computational overhead. Manual verification marks suspicious areas with missing signals in the image, allowing the production system to perform targeted verification based on these marks, preventing erroneous results from silently leaking out.
[0035] In Example 2, the coarse quantization of the gray-level co-occurrence matrix texture statistics in step 3 is obtained by quantizing the image L* channel to 4-16 levels, calculating the local gray-level co-occurrence matrix contrast for multi-directional offsets and normalizing it; the closed hole processing also includes the fine slit dual-signal criterion, background color gate judgment, and exception judgment of the joint between the platform and the background arc; the fine slit dual-signal criterion is: the filling operation is only performed when the texture of the hole erosion area is greater than the texture threshold, and the difference between the hole color and the two background whites is greater than the color difference threshold.
[0036] By adopting the above technical solution, the coarse quantization of gray-level co-occurrence matrix texture statistics is obtained by quantizing the image L* channel to 4 to 16 levels, calculating the local gray-level co-occurrence matrix contrast for multi-directional offsets and normalizing it.
[0037] Coarse quantization of the luminance channel can bring the statistical results within clean, transparent holes close to zero. However, fine quantization can introduce false texture readings due to weak background noise, interfering with hole detection. Multi-directional offset sampling can take into account horizontal, vertical, and diagonal texture features, and normalization processing provides a uniform scale for the statistics.
[0038] The closed hole processing also includes dual-signal criteria for narrow gaps, background color door judgment, and exception judgment for the curved joint between the tabletop and the background.
[0039] When the slit size is too small, morphological erosion operations cannot obtain a pure internal area, and relying solely on texture signals is prone to misjudgment. The dual-signal criterion for slits is that a filling operation is only performed when the texture of the eroded area of the hole is greater than the texture threshold, and the difference between the hole color and the white of both backgrounds is greater than the color difference threshold.
[0040] Texture signals only indicate the presence of texture in a region; they cannot distinguish whether the texture originates from a product component or boundary contamination. After overlaying the color difference condition, only when both the texture and color difference conditions are met simultaneously is it determined to be a small product component for filling, thus avoiding incorrectly filling the gaps in the actual shoelaces.
[0041] Background color doors are used to identify the shadows of the support frame and the seams of the tabletop when viewed from inside the holes. An exception, the curved seam between the tabletop and the background, is used to identify the projected curve of the circular tabletop seen through the gaps in the holes; these types of holes should remain transparent and not be filled.
[0042] In Example 3, the first background white and the second background white are automatically calibrated by the median value of the pixels at the corners of the image and the median value of the pixels in the horizontal band below the product outline, or are obtained by factory pre-calibration of the photographic device.
[0043] By adopting the above technical solution, the first background white and the second background white are automatically calibrated by the median value of the pixels at the corners of the image and the median value of the pixels in the horizontal band below the product outline, or are obtained by the factory pre-calibration of the photographic device.
[0044] The corners of the image are covered by the background; the median of the corner pixels yields the first background white. A horizontal sampling band is defined below the lower edge of the product outline, confined to the horizontal range of the product object. After removing foreground pixels, the median value is used to obtain the second background white. If no sampling band exists, the first background white is reused directly.
[0045] It can also directly read the color values of the two types of background white based on the factory calibration parameters of the photographic device, without relying on sampling calculations based on a single image, making it suitable for equipment with stable and fixed lighting conditions.
[0046] Example 4, step 4, the hole repair process includes a closed ring hole opening mechanism, a confidence wall hole opening mechanism, and a retained hole trimming mechanism; Closed-loop aperture mechanism: Candidate regions that are surrounded by high-confidence solid foregrounds, have low alpha, flat interiors, and areas reaching a threshold are forcibly set to transparent; Confidence wall opening mechanism: Using a verified transparent core as a seed, real through holes are obtained by growing outward from the surrounding high-confidence product wall; Preservation of hole trimming mechanism: Using the color of the corroded area inside the hole as a local background reference, the background debris attached to the hole edge is removed by growing outward under multiple conditions.
[0047] By adopting the above technical solutions, step 4, the hole repair process, includes a closed-loop hole opening mechanism, a confidence wall hole opening mechanism, and a reserved hole trimming mechanism.
[0048] The closed-loop aperture mechanism forces candidate regions that are surrounded by high-confidence solid foregrounds, have low alpha, flat interiors, and areas that reach a threshold to be transparent.
[0049] In a white-on-white scenario, the neural network hesitates to identify real holes, resulting in a lower alpha value in the corresponding area. The hole is completely surrounded by a high-confidence product entity, and there are no texture features inside. Only when multiple conditions—topology, alpha, texture, and area—are met simultaneously can it be identified as a real, blurred hole. Solid product bodies and small textured holes in fabric will not trigger this mechanism.
[0050] The confidence wall opening mechanism uses a verified transparent core as a seed and grows outward from the surrounding high-confidence product wall to obtain a real through hole.
[0051] This mechanism is adapted to holes with partially transparent cores surrounded by high-confidence product pixels. Growth proceeds outward from the confirmed transparent area, with the high-confidence product pixels serving as the growth termination boundary. If no effective confidence wall exists, and the hole area's color is close to the background, representing a white-on-white blind zone with no effective signal, the process stops, and no speculative modifications are made.
[0052] The hole trimming mechanism is retained, and the color of the corroded area inside the hole is used as a local background reference. Multiple conditions are used to constrain outward growth and remove background debris attached to the hole edge.
[0053] The interior of the hole has been confirmed as a perspective background. Compared to global sampling at the four corners of the image, local sampling can adapt to local lighting changes. During the outward growth process, texture increases and colors change. Diffusion stops when any of these conditions occur in the high-confidence pixels. Only background fragments welded to the edge of the hole are removed, without cutting into the product's main pixels.
[0054] Example 5, step 5 monocular depth estimation correction also includes image-by-image self-calibration of the depth band, including the following steps: Catastrophic missing filling, simultaneously satisfying multiple conditions such as convex hull deficiency, hesitant pixel ratio, severity area gate, and being within the depth band to recover lost foreground regions; Background clipping trimming: When all three pieces of evidence—depth location, background color, and background depth level—are met, the mistakenly included background area is trimmed. The veto is reserved; a reserved hole will only be filled if both conditions are met simultaneously. Depth estimation does not overturn the three-state decision of the hole throughout the entire process.
[0055] By adopting the above technical solution, step 5, monocular depth estimation and correction, also includes image-by-image self-calibration of depth bands. Instead of using a globally fixed depth constant, the depth band interval is calculated based on the depth distribution of the foreground elements within the image itself, thus adapting to variations in different shooting objects and distances.
[0056] The correction process consists of three parts of processing logic.
[0057] Catastrophic missing filling, simultaneously satisfying multiple conditions such as convex hull deficiency, hesitant pixel ratio, severity area gate, and being within the depth band to recover lost foreground regions.
[0058] Convex hull deficiency indicates that the contour has a missing area compared to the product's convex hull. The percentage of pixels is used to identify areas with low confidence in the neural network. The severity area gate filters out very small background noise fragments; recovery is only initiated when the missing component reaches a certain area. Components that can be distinguished by color are not directly filled; they are nominated as unknown areas in the triangulation and further determined by the full-resolution image.
[0059] Background clipping trimming: Remove mistakenly included background areas when all three criteria of depth location, background color, and background depth level are met.
[0060] Some background clumps are mistakenly classified as foreground by the segmentation network. Only areas whose depth is further away from the product's depth band, whose color matches the white background, and whose depth is consistent with the actual background paper are pruned. Morphological opening is performed before pruning to remove only clumps without eroding the product's edges.
[0061] The veto is reserved, and the reserved hole will only be filled if both conditions are met.
[0062] For holes selected for retention in the previous steps, the hole alpha must be within a medium confidence range, and most pixels must fall within the depth band before the retention result can be rejected. Real holes with very low confidence in the network output are not affected by this logic, thus protecting special structures such as tunnel-type vias.
[0063] Depth estimation does not overturn the three-state decision of holes throughout the process. The depth signal only participates in the modification of the outer contour and does not change the hole determination result of the fill retention mark. All modification actions require multiple evidences to be triggered together; if the conditions are not met, no pixel changes are performed.
[0064] Example 6, Step 6 edge refinement adopts the precise mode: after upsampling the image, a three-part image is constructed, and the three-part image matting network is run along the contour to solve the unknown region Alpha; a topology protection region is set to force the foreground to remain solid and original in color within the protection region, the edge of the hole retains the anti-aliasing effect, and the foreground is cleaned and the white edge is eliminated only in the narrow edge band.
[0065] By adopting the above technical solution, step 6 edge refinement adopts the precise mode, after upsampling the image to construct a three-part image, and runs the three-part image matting network along the contour to solve the unknown region Alpha.
[0066] The alpha result obtained from the working resolution is bilinearly upsampled to the full resolution size. A three-part image is used to define the foreground, background, and unknown bands around the contour. A block-based solution strategy is employed, with each computational block accompanied by its surrounding context. Only the result of the block's central region is written back, and the results between blocks are naturally connected without additional feathering or stitching. Only blocks containing unknown pixels are fed into the network for computation, reducing computational overhead.
[0067] Set a topology protection area, force the foreground to remain solid and original in color within the protection area, retain anti-aliasing effect at the edge of the aperture, and remove white edges only in the narrow edge band.
[0068] The topology protection region comprises pixels inside the product that are far from holes and the external background. The alpha of this region is forcibly set to 1, preserving the original pixel color to prevent the product's interior, already repaired upstream, from being reprocessed into semi-transparent areas by the triangulation network. A certain width transition band is maintained around the holes to maintain anti-aliasing. Foreground desmearing operations are only performed on the narrow edge bands of the semi-transparent areas, eliminating background color blending into the edges and resolving the white edge phenomenon that appears after compositing.
[0069] Example 7 also includes a switchable fast mode, which uses guided filtering to fit the edges of the full-resolution image and combines it with hierarchical curve mapping to eliminate the low alpha tail on the background side, thereby eliminating white edges and improving processing speed.
[0070] By adopting the above technical solutions, including a switchable fast mode, guided filtering is used to fit the edges of the full-resolution image, and hierarchical curve mapping is used to eliminate the low alpha tail on the background side, thereby eliminating white edges and improving processing speed.
[0071] The fast mode no longer uses the triangulation network; instead, it relies on guided filtering to match the upsampled alpha edges to the original image's true boundaries. Hierarchical curve mapping lowers the weak alpha values on the background side, raises the alpha of the main product, and preserves a smooth transition in the middle area. Fast mode has lower computational overhead, making it suitable for high-volume, high-speed production scenarios, and can be switched with precise mode with a single click based on business needs.
[0072] Example 8: The longest side of the working resolution copy is set to 1536-4096 pixels. All semantic and topological discrimination operations are completed in the working resolution copy, and only the edge thinning operation is performed in the narrow contour band of the full resolution image.
[0073] By adopting the above technical solution, the longest side of the working resolution copy is set to 1536 to 4096 pixels. All semantic and topological discrimination operations are completed in the working resolution copy, and only the edge thinning operation is performed in the narrow contour band of the full resolution image.
[0074] The working resolution setting balances the ability to detect fine seams with the amount of video memory used. If the resolution is too low, small details like shoelaces and pull tabs will become blurred in white-on-white scenes. The high-resolution original image is only saved; hole detection, connected component operations, texture statistics, and depth correction are all performed at reduced resolution. Only a very small area of pixels around the outline is pulled up to full resolution for edge detection, significantly reducing the peak memory usage for ultra-high-resolution image processing.
[0075] Example 9: Modification decisions require at least two independent pieces of evidence; if the conditions are not met, the corresponding changes will not be implemented; manual verification marks carry hole type, area ratio, texture value, color difference, and bounding box structured information.
[0076] By adopting the above technical solution, all modifying decisions require at least two independent pieces of evidence, and if the conditions are not met, the corresponding changes will not be implemented.
[0077] This method does not rely on a single signal to modify pixels. No single channel of color, texture, alpha confidence, or depth can directly drive the hole filling and trimming action. If any condition is not met, the corresponding processing logic skips the process, minimizing erroneous modifications caused by single-channel noise.
[0078] The manual verification markers carry information such as hole type, area ratio, texture value, color difference, and bounding box structure.
[0079] When an image region lacks valid pixel signals, the algorithm cannot provide a definite result and will not force a segmentation result. Instead, it will output structured label information. After reading the labels, the production system can locate suspicious regions and conduct targeted manual verification, avoiding the silent output of unreliable segmentation results.
[0080] Example 10: A product image segmentation system for an automatic photography device, used to implement a product image segmentation method for an automatic photography device, the system comprising: The dual-resolution loading module is used to acquire product images, save a full-resolution copy, and generate a working resolution copy. The soft mask generation module is used to call the matting neural network to generate a soft foreground alpha image and output a set of basic contours and closed holes. The hole tri-state discrimination module is used to combine coarse-quantized gray-level co-occurrence matrix texture statistics and double-background white difference to output a tri-state decision of filling, retaining, and manually verifying the marking of closed holes; The white hole repair module is used to perform through hole repair for closed loop opening, confidence wall opening, and retaining hole trimming; The depth-assisted outer contour correction module is used to run a monocular depth estimation network to correct the outer contour under multi-evidence gating without altering the three-state decision of the hole. The full-resolution edge refinement module is used to perform edge solving, topology protection, and foreground desmearing in narrow contour bands, and outputs an RGBA image. The review mark output module is used to output manual review marks carrying structured information.
[0081] By adopting the above technical solution, the dual-resolution loading module is used to acquire product images, save a full-resolution copy, and generate a working resolution copy. This module completes image reading, the generation and storage of two images, and provides corresponding resolution inputs for subsequent different stages.
[0082] The soft mask generation module calls the image matting neural network to generate a soft foreground alpha image, outputting a set of basic contours and closed holes. This module completes network inference, binarization, and connected component cleanup, extracting all closed holes through image interpolation to provide input for hole detection.
[0083] The hole tri-state discrimination module combines coarse-quantized gray-level co-occurrence matrix texture statistics and double-background white difference to output a tri-state decision on whether to fill, retain, or manually verify closed holes. This module performs background white calibration, calculates texture statistics for each hole, executes the entire set of judgment logic for slits, background color gates, and curved lines as exceptions, and outputs the tri-state result corresponding to each hole.
[0084] The white-on-white hole repair module is used to perform through-hole repairs such as closed-loop hole opening, confidence wall hole opening, and hole trimming. This module reads the three-state discrimination results and performs three types of repair mechanisms for white-on-white glued holes, while adhering to the internal stopping constraints of each mechanism to avoid accidental modification of the product entity.
[0085] The depth-assisted outer contour correction module is used to run a monocular depth estimation network to correct the outer contour under multi-evidence gating without altering the three-state decision of holes. This module performs depth band self-calibration, catastrophic missing object filling, background mis-snagging trimming, retains rejection logic, strictly limits the depth to only apply to the outer contour, and uses multi-path evidence verification to prevent misoperation.
[0086] The full-resolution edge refinement module is used to solve edges, perform topology protection, and foreground desmearing in narrow contour bands, outputting an RGBA image. This module supports switching between precise and fast modes, performs image upsampling, triangulation or guided filtering, executes topology protection constraints, and edge desmearing, ultimately generating an RGBA format image.
[0087] The verification mark output module is used to output manually verified marks carrying structured information. This module summarizes all parameters of the marked holes and outputs structured records, which can be read by upper-level business systems to locate the areas to be verified.
[0088] The following specific embodiments illustrate the implementation principle of the present invention: This case study runs on a consumer-grade GPU workstation. The hardware can utilize Apple's M-series unified memory hardware platform or a general-purpose GPU with CUDA computing capabilities. This case study uses three networks to complete the entire image processing workflow. The first network, BiRefNet_HR, with a training resolution of 2048, is used to generate the soft foreground alpha image. The second network, ViTMatte-small, is used to solve for sub-pixel transparency in unknown regions of the triangulation image. The third network, DepthAnythingV2Small, is responsible for outputting the relative depth information of the image. The entire method is not bound to any specific network; the networks only need to meet the preset interface contract, and can be replaced with other networks with equivalent output capabilities during actual deployment.
[0089] The entire system uses factory-optimized parameters validated with a dataset of 168 samples. All parameters are also set with adjustable ranges to facilitate adaptation to varying lighting conditions and shooting equipment. The working resolution is set to 2048, with an adjustable range of 1536 to 4096. This parameter is constrained by hardware memory; a resolution lower than 2048 will reduce the ability to recognize small gaps in white-on-white scenes. The alpha binarization threshold is set to 0.5, with an adjustable range of 0.3 to 0.7, completing the conversion from soft alpha images to binary basic contours. GLCM quantization levels are set to 8, with an adjustable range of 4 to 16. Coarse quantization allows clean apertures to achieve near-zero statistical readings; excessively fine quantization levels can be compromised by lighting noise. The fill texture gate is set to 0.02, the marker area gate to 0.010, and the slit texture gate and slit color difference gate to 1.5 and 30 respectively. The background color gate is set to 25, and the noise aperture area is set to 20 pixels. The Alpha gate, flatness gate, and area gate for closed-loop apertures are set to 0.96, 0.5, and 80 pixels, respectively. The trimming color difference, confidence gate, and growth upper limit for retained aperture trimming are 14, 0.92, and 6, respectively. Depth band self-calibration uses a core erosion of 30 pixels, with percentiles of 4 and 96. Band margin is composed of a relative value of 0.12 superimposed with an absolute value of 0.04. Parameters related to catastrophe fill, background trimming, retain / reject, three-part image construction, block solving, topology protection, and fast-mode guided filtering are all factory-optimized, while retaining corresponding engineering adjustment ranges.
[0090] Processing a 60-megapixel original product image, the time consumed in each step is as follows: image loading 0.15 seconds, first network inference to generate a soft alpha image 4.5 seconds, hole detection and repair processing 0.26 seconds, depth correction 0.8 seconds, and the third network supports lazy loading, so this step has no time overhead when the depth module is turned off. Edge refinement of the three-part image takes 6 seconds, and writing the RGBA image takes 0.75 seconds. A representative product sample was selected for testing. Three closed holes were detected in the image. The texture statistics inside all holes were close to zero, and all were judged to be retained. The reference intersection-over-union ratio for hole segmentation reached 0.9953, the boundary F-value was 0.9928, the false negative rate was 0.02%, and the false positive rate was 0.20%.
[0091] Appendix Figure 4 This invention includes multiple sets of footwear product test samples. Each set consists of original photographed images arranged sequentially, along with the segmentation output obtained using the method of this invention, and a reference result output by a commercial image cutout service. The test samples cover various challenging white-to-white scenarios, including white shoes, dark shoes, and different shooting angles. The comparison shows that commercial image cutout services suffer from missing parts and incorrect hole processing. The method of this invention can completely preserve the main structure of the product, distinguish between real cutouts and fabric texture holes, and output clean edges, avoiding the problem of silently outputting incorrect segmentation results.
[0092] Repair of white-on-white closed-ring opening: High-resolution images of white trail running shoes were obtained. The heel pull tab is a genuine perforation. Due to the white-on-white condition, the first network inference output blurred this perforation. The alpha distribution of this area was between 0.5 and 0.95, forming an over-containment error region of 4750 pixels. The closed-loop perforation mechanism detected that this area was completely surrounded by a high-confidence solid shoe body, with a flat internal texture, and its area met the threshold condition. Therefore, this area was forcibly set to transparent, reducing the over-containment pixels from 4750 to 86.
[0093] Within the same image, shoelace holes surrounded by white webbing exhibited a similar blurring phenomenon. The confidence wall aperture mechanism was activated, using a verified transparent core as the growth seed and the surrounding white webbing as a high-confidence growth boundary to expand outwards. After correction, the over-containment pixels decreased from 3921 to 298. No false apertures were observed in beige shoes, black shoes, mesh shoes, or white leather boots processed in the same batch. Fabric texture holes did not meet the area triggering condition, and solid shoe uppers did not produce alpha suppression; therefore, neither type of object triggered the aperture logic.
[0094] Implementation of Fine-Seam Dual-Signal Criterion for Small Component Restoration: An image contains footwear with a gray zipper pull. The gap around the zipper pull is very small, and morphological erosion cannot obtain a pure internal area, thus triggering the fine-seam dual-signal judgment process. The GLCM average value of the one-pixel eroded area is 3.7, exhibiting obvious texture features. The color difference between this area and the white of two studio locations reaches 44. Both conditions are met simultaneously, and the system performs a filling operation to completely restore the zipper pull component.
[0095] As a comparison, the texture readings of the eroded areas in the actual shoelace seams within the image also ranged from 3.7 to 4.7. However, the color difference between the interior of the seam and the studio white was less than or equal to 20, failing the color difference condition, and the system did not perform filling processing. Relying solely on the texture signal for judgment would result in a large number of false fills; the measured number of false fill pixels increased from 1106 to 2775. The color difference condition constitutes a reliable separation criterion; therefore, for scenes with narrow seams, both texture and color difference signals must be verified simultaneously.
[0096] Implementation of Depth-Assisted Outer Contour Correction: This case demonstrates three types of correction actions from the depth module, showcasing its zero-side-effect characteristic. The first type is catastrophic missing fill, where the white boot area is largely missed by the image cutout network. The convex hull deficiency component has 108,000 pixels, accounting for 11% of the overall product area, with 0.86 pixels inside the component. Since the component is within the depth band, all conditions are met, and the system recovers the lost foreground area. The maximum false-detection background component in the same image is 4,800 pixels, accounting for less than 1.2%, which does not reach the severity area gate and will not be incorrectly filled.
[0097] The second category involves background cropping errors. In this case, the image shows a recessed area where some background paper is mistakenly classified as foreground. This area's depth is further away from the product's depth band, its color matches the white background, and its depth value is consistent with the background paper's depth level. All three pieces of evidence are met, and the system removes the 8288 pixels of erroneous foreground. In the same image, there is an overexposed fur edge with only a slight depth shift of 0.03 to 0.14, which does not meet all the criteria and is therefore retained. In this scenario, color and texture alone are insufficient for differentiation; depth information becomes the only valid criterion.
[0098] The third category is retention rejection. An incorrect opening mechanism opens the area along the white sole edge. The alpha of this hole is between 0.45 and 0.60, which is considered a medium confidence output in the model. More than 50% of the pixels fall within the depth band. Both conditions are met simultaneously, so the system rejects the incorrect state and restores solid filling. Using a real perforation as a reference sample, the model output alpha is between 0.00 and 0.07. Even if there are supports or shoe bed components at the product's depth behind the hole, it will not be incorrectly sealed because the first condition is not met.
[0099] Statistics on 40 core test samples show that all conditions of the depth module could not be met simultaneously in 36 of the images. Without changing any pixels, the module achieves zero side effects.
[0100] Fast Edge Mode Alternative Implementation: For high-volume, high-speed production scenarios, the system can switch to Fast Edge Mode, eliminating the need for the three-part image matting network. Guided filtering is used to fit the true edges of the full-resolution image, combined with hierarchical curve mapping to eliminate low-value alpha tails on the background side. Processing a single 60-megapixel image takes approximately 6.6 seconds, while the precise mode takes 12 seconds. On the core test dataset, both modes had zero missed samples exceeding one percent. Fast Mode generates edges with a slightly harsher appearance, similar to commercial matting output styles, while the precise mode provides a smoother, more natural edge transition. One-click switching between the two processing modes allows for flexible selection based on production capacity pressure and image quality requirements.
[0101] Multi-scenario generalization and production deployment: This method does not bind to the specific structure of footwear products. The first network completes the general product matting, and the subsequent discrimination process only relies on double white backgrounds, closed hole topology, texture statistics, and depth statistics. In addition to footwear samples, common items such as mugs and electric drills were selected for sampling tests. The segmentation output results were normal, and no category-related failures occurred. Non-sports shoe samples such as bracelets and leather boots in the test dataset also passed the validation. No systematic benchmark evaluation of all categories was carried out.
[0102] For simple studio environments with only one type of white background, the second color reference in the dual-white background criterion can be removed, allowing the entire method to operate at a lower resolution. The original image resolutions that have been validated cover 24 to 60 megapixels, and cross-device stability was verified using 93 samples acquired with a second camera.
[0103] The entire process employs an interface contract constraint on the networks, allowing for free replacement. The first network can be replaced with other high-resolution binary segmentation networks that output soft alpha. The second network can be replaced with a traditional closed-loop matting algorithm, resulting in smoother edge transitions. The third network can be replaced with any other monocular depth estimation network.
[0104] Since July 2026, this solution has been integrated into the automated image output pipeline of intelligent all-in-one cameras, running entirely locally without outputting image data externally. Batch processing jobs employ a strategy of switching processes every 20 images processed, ensuring controllable memory usage. The processing status is recorded using JSONL format, supporting breakpoint recovery.
[0105] Comparison of technical routes rejected by actual tests: This case demonstrates the non-obviousness of the present invention through multiple sets of comparative tests.
[0106] The first comparison route directly relies on color sculpting masks to process the image. The pixel color of the white shoe body is very close to that of the background. After running, the white shoe body parts will be directly deleted, so this method is unusable.
[0107] The second comparison route uses background difference combined with region growing. The pixel amplitude distribution of light-colored product parts and shadow areas overlaps with each other, and there is no threshold that can distinguish them. The algorithm will flood into the shadow area before recognizing the light-colored product.
[0108] The third comparison method directly relies on depth information to determine whether a hole should be filled or left intact. Since the shoe bed and the tabletop face the same direction, their depth values are similar. Neither the depth nor the normal can provide a distinguishing signal. This is why this system limits the depth to only the outer contour and does not participate in the hole determination.
[0109] The fourth comparison route uses surface normals for segmentation. The normal characteristics of the upward-facing inner surface of the shoe bed and the upward-facing platform are the same, making it impossible to distinguish between the two types of areas.
[0110] The fifth comparison route introduces Canny edge assistance, but the calculated features and guided filtering functions are redundant. The number of effective edges decreases significantly in white-on-white scenes, and it fails precisely in the most difficult-to-handle scenes.
[0111] The sixth comparison route uses hard binary edges and then performs white edge removal, resulting in rough output edges. This invention uses hierarchical curves to achieve white edge elimination without sacrificing performance.
[0112] The seventh comparison route involves re-running the matting network for each hole by magnifying the image, which can easily cause small gaps to be filled incorrectly. Repeated calls to the network can also lead to huge computational overhead.
[0113] The eighth comparison route directly feeds the entire 60-megapixel original image into the network. Due to memory limitations, the network input size is limited to approximately 2048 pixels. Direct inference with high-resolution original images results in blurred edges and slow processing speed. A dual-resolution architecture can solve this type of problem. ☝ The ninth comparison route attempts to adjust the binarization threshold to find the missed detection area. The alpha of the missed detection area in the cavity is as low as 0.05. No matter whether the threshold is increased or decreased, the lost part cannot be recovered. It is necessary to rely on a deep retrieval mechanism of multi-evidence fusion.
[0114] The tenth comparison method uses a third-party green screen neural keyer. After processing, the color of the solid area of the product changes, and the median RGB value of the pixel changes from 200 to 234. E-commerce product images have strict requirements for color fidelity, so this method cannot be implemented.
[0115] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.
Claims
1. A method for segmenting product images using an automatic photographic device, characterized in that, Includes the following steps: Step 1: Acquire product images captured by an automatic photography device. The photography device has a background component with a first white background and a product support platform with a second white background. Retain a full-resolution copy of the image and generate a working resolution copy. Step 2: Input the working resolution copy into the trained matting neural network to generate a soft foreground Alpha image. After binarization and connected component cleanup, the basic contour is obtained, and all closed holes corresponding to the basic contour are extracted. Step 3: For each closed hole, combine the coarse gray-level co-occurrence matrix texture statistics inside the hole and the color difference between the hole color and the first white background and the second white background, respectively, and output a three-state decision: fill, retain, and manually check mark. Step 4: Based on the three-state decision processing, close the holes and perform hole repair processing on the real through holes on the white surface that are glued by the model. Step 5: The monocular depth estimation constrained by multi-evidence gating is used to perform correction only on the outer contour, and the depth estimation does not change the three-state decision result of the closed hole; Step 6: Perform edge refinement in the narrow contour region of the full-resolution image to generate a full-resolution RGBA segmentation image, and simultaneously output manually verified labels carrying structured information.
2. The method for segmenting product images for automatic photography equipment according to claim 1, characterized in that, In step 3, the coarse quantization of the gray-level co-occurrence matrix texture statistics is obtained by quantizing the image L* channel to 4-16 levels, calculating the local gray-level co-occurrence matrix contrast for multi-directional offsets and normalizing it; the closed hole processing also includes the fine slit dual signal criterion, background color gate judgment, and exception judgment of the joint between the platform and the background arc; the fine slit dual signal criterion is: the filling operation is only performed when the texture of the hole erosion area is greater than the texture threshold, and the difference between the hole color and the two background whites is greater than the color difference threshold.
3. The method for segmenting product images for automatic photography equipment according to claim 2, characterized in that, The first background white and the second background white are automatically calibrated by the median value of the pixels at the corners of the image and the median value of the pixels in the horizontal band below the product outline, or are obtained by factory pre-calibration of the photographic device.
4. The method for segmenting product images for an automatic photographic device according to claim 3, characterized in that, Step 4, the hole repair process, includes a closed-loop hole opening mechanism, a confidence wall hole opening mechanism, and a reserved hole trimming mechanism. Closed-loop aperture mechanism: Candidate regions that are surrounded by high-confidence solid foregrounds, have low alpha, flat interiors, and areas reaching a threshold are forcibly set to transparent; Confidence wall opening mechanism: Using a verified transparent core as a seed, real through holes are obtained by growing outward from the surrounding high-confidence product wall; Preservation of hole trimming mechanism: Using the color of the corroded area inside the hole as a local background reference, the background debris attached to the hole edge is removed by growing outward under multiple conditions.
5. The method for segmenting product images for an automatic photographic device according to claim 4, characterized in that, Step 5, monocular depth estimation correction, also includes image-by-image self-calibration of depth bands, comprising the following steps: Catastrophic missing filling, simultaneously satisfying multiple conditions such as convex hull deficiency, hesitant pixel ratio, severity area gate, and being within the depth band to recover lost foreground regions; Background clipping trimming: When all three pieces of evidence—depth location, background color, and background depth level—are met, the mistakenly included background area is trimmed. The veto is reserved; a reserved hole will only be filled if both conditions are met simultaneously. Depth estimation does not overturn the three-state decision of the hole throughout the entire process.
6. The method for segmenting product images for an automatic photographic device according to claim 5, characterized in that, Step 6 edge refinement uses the precise mode: after upsampling the image, a three-part image is constructed, and the three-part image matting network is run along the contour to solve for the alpha of the unknown region; Set a topology protection area, force the foreground to remain solid and original in color within the protection area, retain anti-aliasing effect at the edge of the aperture, and remove white edges only in the narrow edge band.
7. The method for segmenting product images for an automatic photographic device according to claim 6, characterized in that, It also includes a switchable fast mode that uses guided filtering to fit the edges of full-resolution images, combined with hierarchical curve mapping to eliminate the low alpha tail on the background side, thereby eliminating white edges and improving processing speed.
8. The method for segmenting product images for an automatic photographic device according to claim 7, characterized in that, The longest side of the working resolution copy is set to 1536-4096 pixels. All semantic and topological discrimination operations are completed in the working resolution copy, and only the edge thinning operation is performed within the narrow contour band of the full-resolution image.
9. A method for segmenting product images for an automatic photographic device according to claim 8, characterized in that, Modification decisions require at least two independent pieces of evidence; if the conditions are not met, the corresponding changes will not be implemented. The manual verification markers carry hole type, area ratio, texture value, color difference, and bounding box structure information.
10. A product image segmentation system for an automatic photographic device, characterized in that, The system for implementing the product image segmentation method for an automatic photography device as described in claim 9 comprises: The dual-resolution loading module is used to acquire product images, save a full-resolution copy, and generate a working resolution copy. The soft mask generation module is used to call the matting neural network to generate a soft foreground alpha image and output a set of basic contours and closed holes. The hole tri-state discrimination module is used to combine coarse-quantized gray-level co-occurrence matrix texture statistics and double-background white difference to output a tri-state decision of filling, retaining, and manually verifying the marking of closed holes; The white hole repair module is used to perform through hole repair for closed loop opening, confidence wall opening, and retaining hole trimming; The depth-assisted outer contour correction module is used to run a monocular depth estimation network to correct the outer contour under multi-evidence gating without altering the three-state decision of the hole. The full-resolution edge refinement module is used to perform edge solving, topology protection, and foreground desmearing in narrow contour bands, and outputs an RGBA image. The review mark output module is used to output manual review marks carrying structured information.
