Visual segmentation counting method and system suitable for disorderly stacked parts

Abstract

The invention relates to the field of image processing, and provides a visual segmentation counting method and system suitable for disorderly stacked parts, which can avoid the influence caused by a complex construction environment by carrying out dynamic Gamma correction and overlapping region enhancement processing on part images of multiple visual angles, highlight and enhance the characteristics of the overlapping region, and improve the accuracy of the visual segmentation counting of the disorderly stacked parts. Specific detail information of a shielding part and a small target is easier to recognize, recognition accuracy and robustness are improved, basic semantic segmentation is performed through a lightweight CNN model, recognition precision is guaranteed, meanwhile, calculation amount is greatly reduced, recognition efficiency is improved, and through dynamic threshold segmentation, seed point production and region growth, recognition efficiency is improved. According to the visual segmentation counting method for the disordered and stacked parts, the semantic and geometric features of the image are combined, the shielded and small-size parts are segmented more accurately, the omission ratio is effectively reduced, higher adaptability and detection precision are shown in a complex scene, and the accuracy and efficiency of the visual segmentation counting method for the disordered and stacked parts are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a visual segmentation and counting method and system suitable for randomly stacked components. Background Technology

[0002] With the rapid development of industrial production, the production speed of parts has gradually increased, resulting in a large number of messy stacked parts, which poses a significant challenge to the calculation and inspection of parts production volume.

[0003] In existing technologies, traditional manual counting methods are no longer sufficient to meet the high-efficiency production pace. Not only is counting time-consuming, but the accuracy is also poor. Therefore, image recognition processing algorithms based on computer vision have gradually become the mainstream. However, existing image processing algorithms, such as the watershed algorithm based on edge detection, still have high errors and long processing times when dealing with a large number of parts that are densely occluded (overlap rate greater than 50%). In particular, when the parts are small (pixel area less than 100px²), small targets cause more missed detections, and the robustness is poor, making them susceptible to the influence of complex production counting environments, which leads to a further decrease in counting accuracy.

[0004] Therefore, it is necessary to design a visual segmentation and counting method suitable for messy stacked parts to avoid the influence of dense occlusion and small targets, and improve the accuracy of counting. Summary of the Invention

[0005] Based on this, the present invention proposes a visual segmentation and counting method and system suitable for randomly stacked parts. By performing dynamic Gamma correction and overlapping region enhancement processing on part images from multiple perspectives, the method avoids the influence of complex construction environments and highlights and enhances the features of overlapping regions, making it easier to identify the specific details of occluded parts and small targets, thus improving the accuracy and robustness of recognition. Then, a lightweight CNN model is used for basic semantic segmentation, which significantly reduces the amount of computation and improves the efficiency of recognition while ensuring recognition accuracy. Furthermore, by using dynamic threshold segmentation, seed point generation, and region growing, combined with the semantic and geometric features of the image, the method can more accurately segment occluded and small-sized parts, effectively reducing the false negative rate. It exhibits stronger adaptability and detection accuracy in complex scenes. The present invention improves the accuracy and efficiency of visual segmentation and counting methods for randomly stacked parts.

[0006] This invention proposes a visual segmentation and counting method for randomly stacked components, comprising: Acquire multi-view component images and perform preprocessing. The multi-view component images are based on the top view image, front view image, left view image and right view image of the pipeline detection area. The preprocessing includes dynamic Gamma correction and overlapping region enhancement processing. The multi-view component images are semantically segmented using a lightweight CNN model to obtain a component semantic probability map. The lightweight CNN model is based on depthwise separable convolution and cross-layer connections. The semantic probability graph of the components is dynamically thresholded to obtain a set of target seed points. The dynamic thresholding is based on the mean and standard deviation of the semantic probability graph of the components, and the set of target seed points is based on the centroid of the connected components. The target seed point set is iterated to obtain a component instance set, which is based on a component instance mask. The set of component instances is subjected to overlap separation processing to obtain a valid set of component instances. The number of components is then calculated based on the connected component analysis algorithm. The overlap separation processing is based on the Canny algorithm and the watershed algorithm.

[0007] In summary, based on the aforementioned visual segmentation and counting method applicable to randomly stacked parts, dynamic Gamma correction and overlapping region enhancement processing are applied to part images from multiple perspectives to avoid the influence of complex construction environments. Simultaneously, the features of overlapping regions are highlighted and enhanced, making it easier to identify the specific details of occluded parts and small targets, thus improving the accuracy and robustness of recognition. Furthermore, basic semantic segmentation is performed using a lightweight CNN model, significantly reducing computational load and improving recognition efficiency while maintaining recognition accuracy. Dynamic threshold segmentation, seed point generation, and region growing, combined with the semantic and geometric features of the image, can more accurately segment occluded and small-sized parts, effectively reducing the false negative rate. This invention demonstrates stronger adaptability and detection accuracy in complex scenarios, improving the accuracy and efficiency of visual segmentation and counting methods for randomly stacked parts. Specifically, the process involves acquiring and preprocessing multi-view component images. These images are based on top-view, front-view, left-view, and right-view images of the pipeline detection area. The preprocessing includes dynamic Gamma correction and overlapping region enhancement, which avoids the influence of complex construction environments while highlighting and enhancing the features of overlapping regions. This makes it easier to identify occluded parts and small targets, improving accuracy and robustness. A lightweight CNN model is then used to perform semantic segmentation on the multi-view component images to obtain a component semantic probability map. This lightweight CNN model, based on depthwise separable convolutions and cross-layer connections, significantly reduces computation while maintaining recognition accuracy, improving efficiency. The component semantic probability map is then dynamically segmented. Threshold segmentation is used to obtain a target seed point set. The dynamic threshold segmentation is based on the mean and standard deviation of the semantic probability map of the parts. The target seed point set is based on the centroid of the connected components. The process iterates based on the target seed point set to obtain a set of parts instances. The set of parts instances is based on a part instance mask. Overlap separation processing is performed on the set of parts instances to obtain a valid set of parts instances. The number of parts is calculated according to the connected component analysis algorithm. The overlap separation processing is based on the Canny algorithm and the watershed algorithm. Combining the semantic and geometric features of the image, it can more accurately segment occluded and small-sized parts, effectively reduce the false negative rate, and show stronger adaptability and detection accuracy in complex scenes. This invention improves the accuracy and efficiency of the visual segmentation and counting method for randomly stacked parts.

[0008] Furthermore, the step of acquiring multi-view component images and performing preprocessing specifically includes: Acquire multi-view images of components; Dynamic Gamma correction is performed on the multi-view component images. The specific algorithm for dynamic Gamma correction is as follows: , in, This represents the corrected pixel value. Represents the original pixel value. Indicates dynamic correction parameters. Indicates the average brightness of the image; according to The spatial kernel and the 15-value-range kernel are used to filter the multi-view component images after dynamic Gamma correction in order to filter and retain the edge detail information of the multi-view component images. The filtered multi-view component images are then subjected to overlapping region enhancement processing. The specific algorithm for overlapping region enhancement processing is as follows: , in, This represents the multi-view component images after weighted fusion. , , , These represent the top-view image, front-view image, left-view image, and right-view image of the pipeline detection region. , , , The weights of images from different viewpoints are represented, and these weights are based on the component type.

[0009] Furthermore, the step of performing semantic segmentation processing on the multi-view component images using a lightweight CNN model to obtain component semantic probability maps specifically includes: The multi-view component images are scaled to 512×512 pixels, and then normalized. The specific algorithm for normalization is as follows: , in, This represents the multi-view component images after weighted fusion. This represents the normalized multi-view component images; Normalized multi-view component images are input into a lightweight CNN model, which includes 8 depthwise separable convolutional layers and a cross-layer connection structure to preserve and enhance low-level edge features, outputting a component semantic probability map. This semantic probability map is 128×128 pixels and has [number of channels missing]. The N This indicates the number of component types, including bolts and gears.

[0010] Furthermore, the step of performing dynamic threshold segmentation on the semantic probability map of the components to obtain the target seed point set specifically includes: The semantic probability map of the components is divided into 16×16 regions, each region comprising a 32×32 pixel sub-block. The mean and standard deviation of each sub-block in the semantic probability map of the components are calculated. The specific algorithm for calculating the mean and standard deviation is as follows: , , in, This represents the mean. Indicates standard deviation, Represents the semantic probability graph of components. Indicates the sub-block ordinal number; The dynamic threshold for each sub-block is calculated using the following algorithm: , in, Indicates a dynamic threshold; A binary mask is obtained based on the dynamic threshold and the semantic probability map of the component, and compared and judged. If the semantic probability map of the component in any region is greater than or equal to the dynamic threshold, the value of the binary mask in that region is 1. If the semantic probability map of the component in any region is less than the dynamic threshold, the value of the binary mask in that region is 0. Based on morphological image processing, the binary mask is eroded. The specific algorithm for erosion is as follows: , in, This indicates the mask after etching. Represents a binary mask. express nuclear; The etched mask is labeled with 8-neighbor connected components to calculate the centroid of each connected component. The set of these centroids is then used as the target seed point set. The specific algorithm for calculating the centroids is as follows: , , in, This represents the x-coordinate of the seed point corresponding to the connected component. This represents the x-coordinate of the seed point corresponding to the connected component. This indicates the number of pixels within a connected component. Represents the index ordinal number of the pixel within the connected component. and This represents the x and y coordinates of a pixel within a connected region.

[0011] Furthermore, the step of iterating based on the target seed point set to obtain the component instance set specifically includes: Extract the RGB color value of each seed point in the target seed point set. The RGB color value is used as the reference color for iterative growth. Iterative growth is performed on the seed point. The iterative growth is expanded according to the upper neighbor, lower neighbor, left neighbor and right neighbor of the seed point. The iterative growth stops when the preset maximum number of iterations or the preset maximum number of no update cycles is reached, so as to obtain a component instance set. Each component instance in the component instance set has a unique corresponding component instance mask.

[0012] Furthermore, the step of performing overlap separation processing on the set of component instances to obtain a valid set of component instances specifically includes: The contours of each component instance mask in the component instance set are extracted according to the Canny algorithm to obtain a closed contour set. The minimum pixel distance between any two closed contours in the closed contour set is calculated. The minimum pixel distance is filtered and judged. If the minimum pixel distance is less than 20 pixels, the overlapping separation processing of the component instance mask corresponding to the closed contour is performed based on the Sobel gradient map. The overlapping and separated component instance masks are filtered by area. If the area of ​​the component instance mask is less than a preset mask area threshold, the component instance mask is deleted. The solidity of the remaining component instance masks after deletion is calculated. The specific algorithm for the solidity calculation is as follows: , in, Indicates the degree of solidity. Indicates the mask area. Represents the area of ​​the convex hull; Based on the solidity, a solidity filter is performed. If the solidity of the component instance mask is less than or equal to a preset solidity threshold, the component instance mask is deleted. The deleted component instances are masked to form a collection of valid component instances.

[0013] Furthermore, the step of forming a valid set of component instances from the deleted component instance masks then includes: The connected component analysis algorithm identifies all connected regions in the set of valid component instances. The number of components is calculated based on the number of connected regions. Each connected region has a unique corresponding component. The number of components is divided according to their type.

[0014] This invention proposes a visual segmentation and counting system for randomly stacked components, comprising: The preprocessing module is used to acquire and preprocess multi-view component images. The multi-view component images are based on the top view image, front view image, left view image, and right view image of the pipeline detection area. The preprocessing includes dynamic Gamma correction and overlapping region enhancement processing. The semantic segmentation module is used to perform semantic segmentation processing on the multi-view component images according to a lightweight CNN model to obtain a semantic probability map of the components. The lightweight CNN model is based on depthwise separable convolution and cross-layer connections. The dynamic threshold segmentation module is used to perform dynamic threshold segmentation on the semantic probability map of the components to obtain a set of target seed points. The dynamic threshold segmentation is based on the mean and standard deviation of the semantic probability map of the components, and the set of target seed points is based on the centroid of the connected components. The seed iteration module is used to iterate according to the target seed point set to obtain a component instance set, the component instance set being based on a component instance mask; The separation counting module is used to perform overlapping separation processing on the set of component instances to obtain a valid set of component instances, so as to calculate the number of components according to the connected component analysis algorithm. The overlapping separation processing is based on the Canny algorithm and the watershed algorithm.

[0015] The present invention also provides a storage medium storing one or more programs that, when executed by a processor, implement the visual segmentation and counting method for randomly stacked components as described above.

[0016] The present invention also provides a computer device, the computer device including a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the visual segmentation and counting method for randomly stacked components as described above. Attached Figure Description

[0017] Figure 1 This is a flowchart of a visual segmentation and counting method for randomly stacked components proposed in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a visual segmentation and counting system for randomly stacked components proposed in the second embodiment of the present invention.

[0018] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0019] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0020] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] Please see Figure 1 The diagram shows a flowchart of a visual segmentation and counting method for randomly stacked components proposed in the first embodiment of the present invention. This visual segmentation and counting method for randomly stacked components includes steps S01 to S05, wherein: Step S01: Acquire multi-view component images and perform preprocessing; It should be noted that in this embodiment, the multi-view component images are based on the top view image, front view image, left view image and right view image of the pipeline detection area, and the preprocessing includes dynamic Gamma correction and overlapping region enhancement processing. In this embodiment, four industrial area scan cameras are horizontally installed above and around the inspection station on the industrial production line, positioned in the top, front, left, and right view directions respectively. The top-view camera is 50cm above the stacking table, with its optical axis perpendicular to the table surface. The optical axes of the remaining cameras are parallel to the table surface, and the three cameras are spaced 90 degrees apart. The four cameras can be Basler-acA1920-155um models, with a resolution of 1920×1280 and a frame rate of 15fps. A ring-shaped shadowless light source is arranged below the top camera, and strip light sources are arranged near the left and right view cameras. The ring-shaped shadowless light source is white light, and the strip light source is blue light with a wavelength of 450nm. When the photoelectric sensor at the end of the conveyor belt detects that the component stacking tray is in place, it triggers four cameras to start synchronously via TTL signal, and simultaneously turns on the ring-shaped shadowless light source and the strip light sources on both sides. At the same time, the exposure is completed within 10ms, and finally four RGB images with timestamps are obtained, named Top_Image.jpg, Front_Image.jpg, Left_Image.jpg, and Right_Image.jpg respectively, with a resolution of 1920×1280, and saved to the edge computing unit. Acquire multi-view images of components; Dynamic Gamma correction is performed on the multi-view component images. The specific algorithm for dynamic Gamma correction is as follows: , in, This represents the corrected pixel value. Represents the original pixel value. Indicates dynamic correction parameters. Indicates the average brightness of the image; according to The spatial kernel and the 15-value-range kernel are used to filter the multi-view component images after dynamic Gamma correction in order to filter and retain the edge detail information of the multi-view component images. The filtered multi-view component images are then subjected to overlapping region enhancement processing. The specific algorithm for overlapping region enhancement processing is as follows: , in, This represents the multi-view component images after weighted fusion. , , , These represent the top-view image, front-view image, left-view image, and right-view image of the pipeline detection region. , , , The weights of images from different viewpoints are represented, and these weights are based on the component type.

[0023] Step S02: Perform semantic segmentation processing on multi-view component images using a lightweight CNN model to obtain component semantic probability maps; It should be noted that in this embodiment, the lightweight CNN model is based on depthwise separable convolution and cross-layer connections to scale the multi-view component images. The scaling process adjusts the multi-view component images to 512×512 pixels, and the multi-view component images are then normalized. The specific algorithm for the normalization process is as follows: , in, This represents the multi-view component images after weighted fusion. This represents the normalized multi-view component images; Normalized multi-view component images are input into a lightweight CNN model, which includes 8 depthwise separable convolutional layers and a cross-layer connection structure to preserve and enhance low-level edge features, outputting a component semantic probability map. This semantic probability map is 128×128 pixels and has [number of channels missing]. The N This indicates the number of component types, including bolts and gears.

[0024] Step S03: Perform dynamic threshold segmentation on the semantic probability map of the parts to obtain the target seed point set; It should be noted that in this embodiment, the dynamic threshold segmentation is based on the mean and standard deviation of the semantic probability map of the components, and the target seed point set is based on the centroid of the connected components. The semantic probability map of the components is divided into 16×16 regions, and each region includes a 32×32 pixel sub-block. The mean and standard deviation of each sub-block in the semantic probability map of the components are calculated. The specific algorithm for calculating the mean and standard deviation is as follows: , , in, This represents the mean. Indicates standard deviation, Represents the semantic probability graph of components. Indicates the sub-block ordinal number; The dynamic threshold for each sub-block is calculated using the following algorithm: , in, Indicates a dynamic threshold; A binary mask is obtained based on the dynamic threshold and the semantic probability map of the component, and compared and judged. If the semantic probability map of the component in any region is greater than or equal to the dynamic threshold, the value of the binary mask in that region is 1. If the semantic probability map of the component in any region is less than the dynamic threshold, the value of the binary mask in that region is 0. Based on morphological image processing, the binary mask is eroded. The specific algorithm for erosion is as follows: , in, This indicates the mask after etching. Represents a binary mask. express nuclear; The etched mask is labeled with 8-neighbor connected components to calculate the centroid of each connected component. The set of these centroids is then used as the target seed point set. The specific algorithm for calculating the centroids is as follows: , , in, This represents the x-coordinate of the seed point corresponding to the connected component. This represents the x-coordinate of the seed point corresponding to the connected component. This indicates the number of pixels within a connected component. Indicates the index ordinal number of the pixel within the connected component. and This represents the x and y coordinates of a pixel within a connected region.

[0025] Step S04: Iterate based on the target seed point set to obtain the component instance set; It should be noted that in this embodiment, the component instance set is based on the component instance mask. The RGB color value of each seed point in the target seed point set is extracted. The RGB color value is used as the reference color for iterative growth. The seed point is iteratively grown. The iterative growth is expanded according to the upper neighbor, lower neighbor, left neighbor and right neighbor of the seed point. The iterative growth stops when the preset maximum iteration round or the preset maximum no update round is reached, so as to obtain the component instance set. Each component instance in the component instance set has a unique corresponding component instance mask. In this embodiment, the maximum number of iterations is preset to 50 generations, and the maximum number of no-update cycles is preset to 5 generations. No-update cycles refer to cycles in which no new pixels are added consecutively.

[0026] Step S05: Perform overlap separation processing on the set of component instances to obtain a valid set of component instances, so as to calculate the number of components based on the connected component analysis algorithm; It should be noted that in this embodiment, the overlap separation process is based on the Canny algorithm and the watershed algorithm. According to the Canny algorithm, the contour of each component instance mask in the component instance set is extracted to obtain a closed contour set. The minimum pixel distance between any two closed contours in the closed contour set is calculated. The minimum pixel distance is filtered and judged. If the minimum pixel distance is less than 20 pixels, the overlap separation process of the component instance mask corresponding to the closed contour is performed based on the Sobel gradient map. The overlapping and separated component instance masks are filtered by area. If the area of ​​the component instance mask is less than a preset mask area threshold, the component instance mask is deleted. The solidity of the remaining component instance masks after deletion is calculated. The specific algorithm for the solidity calculation is as follows: , in, Indicates the degree of solidity. Indicates the mask area. Represents the area of ​​the convex hull; Based on the solidity, a solidity filter is performed. If the solidity of the component instance mask is less than or equal to a preset solidity threshold, the component instance mask is deleted. In this embodiment, the preset mask area threshold is 100, and the preset solidity threshold is 0.4; The deleted component instances are masked to form a collection of valid component instances; The connected component analysis algorithm identifies all connected regions in the set of valid component instances. The number of components is calculated based on the number of connected regions. Each connected region has a unique corresponding component. The number of components is divided according to their type.

[0027] In summary, based on the aforementioned visual segmentation and counting method applicable to randomly stacked parts, dynamic Gamma correction and overlapping region enhancement processing are applied to part images from multiple perspectives to avoid the influence of complex construction environments. Simultaneously, the features of overlapping regions are highlighted and enhanced, making it easier to identify the specific details of occluded parts and small targets, thus improving the accuracy and robustness of recognition. Furthermore, basic semantic segmentation is performed using a lightweight CNN model, significantly reducing computational load and improving recognition efficiency while maintaining recognition accuracy. Dynamic threshold segmentation, seed point generation, and region growing, combined with the semantic and geometric features of the image, can more accurately segment occluded and small-sized parts, effectively reducing the false negative rate. This invention demonstrates stronger adaptability and detection accuracy in complex scenarios, improving the accuracy and efficiency of visual segmentation and counting methods for randomly stacked parts. Specifically, the process involves acquiring and preprocessing multi-view component images. These images are based on top-view, front-view, left-view, and right-view images of the pipeline detection area. The preprocessing includes dynamic Gamma correction and overlapping region enhancement, which avoids the influence of complex construction environments while highlighting and enhancing the features of overlapping regions. This makes it easier to identify occluded parts and small targets, improving accuracy and robustness. A lightweight CNN model is then used to perform semantic segmentation on the multi-view component images to obtain a component semantic probability map. This lightweight CNN model, based on depthwise separable convolutions and cross-layer connections, significantly reduces computation while maintaining recognition accuracy, improving efficiency. The component semantic probability map is then dynamically segmented. Threshold segmentation is used to obtain a target seed point set. The dynamic threshold segmentation is based on the mean and standard deviation of the semantic probability map of the parts. The target seed point set is based on the centroid of the connected components. The process iterates based on the target seed point set to obtain a set of parts instances. The set of parts instances is based on a part instance mask. Overlap separation processing is performed on the set of parts instances to obtain a valid set of parts instances. The number of parts is calculated according to the connected component analysis algorithm. The overlap separation processing is based on the Canny algorithm and the watershed algorithm. Combining the semantic and geometric features of the image, it can more accurately segment occluded and small-sized parts, effectively reduce the false negative rate, and show stronger adaptability and detection accuracy in complex scenes. This invention improves the accuracy and efficiency of the visual segmentation and counting method for randomly stacked parts.

[0028] Please see Figure 2 The diagram shows a schematic representation of a visual segmentation and counting system for randomly stacked components according to a second embodiment of the present invention. The system includes: Preprocessing module 10 is used to acquire and preprocess multi-view component images. The multi-view component images are based on the top view image, front view image, left view image and right view image of the pipeline detection area. The preprocessing includes dynamic Gamma correction and overlapping region enhancement processing. The semantic segmentation module 20 is used to perform semantic segmentation processing on the multi-view component images according to a lightweight CNN model to obtain a semantic probability map of the components. The lightweight CNN model is based on depthwise separable convolution and cross-layer connections. The dynamic threshold segmentation module 30 is used to perform dynamic threshold segmentation on the semantic probability map of the component to obtain a target seed point set. The dynamic threshold segmentation is based on the mean and standard deviation of the semantic probability map of the component, and the target seed point set is based on the centroid of the connected component. Seed iteration module 40 is used to iterate according to the target seed point set to obtain a component instance set, the component instance set being based on a component instance mask; The separation counting module 50 is used to perform overlapping separation processing on the set of component instances to obtain a valid set of component instances, so as to calculate the number of components according to the connected component analysis algorithm. The overlapping separation processing is based on the Canny algorithm and the watershed algorithm.

[0029] The present invention also proposes a computer storage medium storing one or more programs that, when executed by a processor, implement the above-described visual segmentation and counting method for randomly stacked components.

[0030] The present invention also proposes a computer device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-described visual segmentation and counting method for randomly stacked components.

[0031] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain stored, communicated, propagated, or transmitted programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0032] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0033] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0034] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0035] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A visual segmentation and counting method suitable for randomly stacked parts, characterized in that, include: Acquire multi-view component images and perform preprocessing. The multi-view component images are based on the top view image, front view image, left view image and right view image of the pipeline detection area. The preprocessing includes dynamic Gamma correction and overlapping region enhancement processing. The multi-view component images are semantically segmented using a lightweight CNN model to obtain a component semantic probability map. The lightweight CNN model is based on depthwise separable convolution and cross-layer connections. The semantic probability graph of the components is dynamically thresholded to obtain a set of target seed points. The dynamic thresholding is based on the mean and standard deviation of the semantic probability graph of the components, and the set of target seed points is based on the centroid of the connected components. The target seed point set is iterated to obtain a component instance set, which is based on a component instance mask. The set of component instances is subjected to overlap separation processing to obtain a valid set of component instances. The number of components is then calculated based on the connected component analysis algorithm. The overlap separation processing is based on the Canny algorithm and the watershed algorithm.

2. The visual segmentation and counting method for randomly stacked components according to claim 1, characterized in that, The steps of acquiring multi-view component images and performing preprocessing specifically include: Acquire multi-view images of components; Dynamic Gamma correction is performed on the multi-view component images. The specific algorithm for dynamic Gamma correction is as follows: ， in, This represents the corrected pixel value. Represents the original pixel value. Indicates dynamic correction parameters. Indicates the average brightness of the image; according to The spatial kernel and the 15-value-range kernel are used to filter the multi-view component images after dynamic Gamma correction in order to filter and retain the edge detail information of the multi-view component images. The filtered multi-view component images are then subjected to overlapping region enhancement processing. The specific algorithm for overlapping region enhancement processing is as follows: ， in, This represents the multi-view component images after weighted fusion. , , , These represent the top-view image, front-view image, left-view image, and right-view image of the pipeline detection region. , , , The weights of images from different viewpoints are represented, and these weights are based on the component type.

3. The visual segmentation and counting method for randomly stacked components according to claim 1, characterized in that, The step of performing semantic segmentation processing on the multi-view component images using a lightweight CNN model to obtain component semantic probability maps specifically includes: The multi-view component images are scaled to 512×512 pixels, and then normalized. The specific algorithm for normalization is as follows: ， in, This represents the multi-view component images after weighted fusion. This represents the normalized multi-view component images; Normalized multi-view component images are input into a lightweight CNN model, which includes 8 depthwise separable convolutional layers and a cross-layer connection structure to preserve and enhance low-level edge features, outputting a component semantic probability map. This semantic probability map is 128×128 pixels and has [number of channels missing]. The N This indicates the number of component types, including bolts and gears.

4. The visual segmentation and counting method for randomly stacked components according to claim 1, characterized in that, The step of performing dynamic threshold segmentation on the semantic probability map of the components to obtain the target seed point set specifically includes: The semantic probability map of the components is divided into 16×16 regions, each region comprising a 32×32 pixel sub-block. The mean and standard deviation of each sub-block in the semantic probability map of the components are calculated. The specific algorithm for calculating the mean and standard deviation is as follows: ， ， in, This represents the mean. Indicates standard deviation, Represents the semantic probability graph of components. Indicates the sub-block ordinal number; The dynamic threshold for each sub-block is calculated using the following algorithm: ， in, Indicates a dynamic threshold; A binary mask is obtained based on the dynamic threshold and the semantic probability map of the component, and compared and judged. If the semantic probability map of the component in any region is greater than or equal to the dynamic threshold, the value of the binary mask in that region is 1. If the semantic probability map of the component in any region is less than the dynamic threshold, the value of the binary mask in that region is 0. Based on morphological image processing, the binary mask is eroded. The specific algorithm for erosion is as follows: ， in, This indicates the mask after etching. Represents a binary mask. express nuclear; The etched mask is labeled with 8-neighbor connected components to calculate the centroid of each connected component. The set of these centroids is then used as the target seed point set. The specific algorithm for calculating the centroids is as follows: ， ， in, This represents the x-coordinate of the seed point corresponding to the connected component. This represents the x-coordinate of the seed point corresponding to the connected component. This indicates the number of pixels within a connected component. Indicates the index ordinal number of the pixel within the connected component. and This represents the x and y coordinates of a pixel within a connected region.

5. The visual segmentation and counting method for randomly stacked components according to claim 1, characterized in that, The step of iterating based on the target seed point set to obtain the component instance set specifically includes: Extract the RGB color value of each seed point in the target seed point set. The RGB color value is used as the reference color for iterative growth. Iterative growth is performed on the seed point. The iterative growth is expanded according to the upper neighbor, lower neighbor, left neighbor and right neighbor of the seed point. The iterative growth stops when the preset maximum number of iterations or the preset maximum number of no update cycles is reached, so as to obtain a component instance set. Each component instance in the component instance set has a unique corresponding component instance mask.

6. The visual segmentation and counting method for randomly stacked components according to claim 1, characterized in that, The step of performing overlap separation processing on the set of component instances to obtain a valid set of component instances specifically includes: The contours of each component instance mask in the component instance set are extracted according to the Canny algorithm to obtain a closed contour set. The minimum pixel distance between any two closed contours in the closed contour set is calculated. The minimum pixel distance is filtered and judged. If the minimum pixel distance is less than 20 pixels, the overlapping separation processing of the component instance mask corresponding to the closed contour is performed based on the Sobel gradient map. The overlapping and separated component instance masks are filtered by area. If the area of ​​the component instance mask is less than a preset mask area threshold, the component instance mask is deleted. The solidity of the remaining component instance masks after deletion is calculated. The specific algorithm for the solidity calculation is as follows: ， in, Indicates the degree of solidity. Indicates the mask area. Represents the area of ​​the convex hull; Based on the solidity, a solidity filter is performed. If the solidity of the component instance mask is less than or equal to a preset solidity threshold, the component instance mask is deleted. The deleted component instances are masked to form a collection of valid component instances.

7. The visual segmentation and counting method for randomly stacked components according to claim 6, characterized in that, The step of forming a valid set of component instances from the mask of the deleted component instances then includes: The connected component analysis algorithm identifies all connected regions in the set of valid component instances. The number of components is calculated based on the number of connected regions. Each connected region has a unique corresponding component. The number of components is divided according to their type.

8. A visual segmentation and counting system suitable for randomly stacked parts, characterized in that, include: The preprocessing module is used to acquire and preprocess multi-view component images. The multi-view component images are based on the top view image, front view image, left view image, and right view image of the pipeline detection area. The preprocessing includes dynamic Gamma correction and overlapping region enhancement processing. The semantic segmentation module is used to perform semantic segmentation processing on the multi-view component images according to a lightweight CNN model to obtain a semantic probability map of the components. The lightweight CNN model is based on depthwise separable convolution and cross-layer connections. The dynamic threshold segmentation module is used to perform dynamic threshold segmentation on the semantic probability map of the components to obtain a set of target seed points. The dynamic threshold segmentation is based on the mean and standard deviation of the semantic probability map of the components, and the set of target seed points is based on the centroid of the connected components. The seed iteration module is used to iterate according to the target seed point set to obtain a component instance set, the component instance set being based on a component instance mask; The separation counting module is used to perform overlapping separation processing on the set of component instances to obtain a valid set of component instances, so as to calculate the number of components according to the connected component analysis algorithm. The overlapping separation processing is based on the Canny algorithm and the watershed algorithm.

9. A storage medium, characterized in that, The storage medium stores one or more programs that, when executed by a processor, implement the visual segmentation and counting method for randomly stacked components as described in any one of claims 1-7.

10. A computer device, characterized in that, The computer device includes a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the visual segmentation and counting method for randomly stacked components as described in any one of claims 1-7.

Images