A ton bag robust visual detection system based on multi-cluster feature fusion and posture correction driving

Abstract

The present application relates to the technical field of machine vision, in particular to a kind of robust visual detection system of ton bag goods based on multi-cluster feature fusion and posture correction drive, the system includes non-local mean enhancement module, multi-feature clustering fusion module, geometric feature extraction module, dominant posture clustering module, posture correction transformation module and goods segmentation determination module, non-local mean filtering is carried out to the original image of ton bag goods to obtain enhanced image, the present application is equipped with non-local mean enhancement module and multi-feature clustering fusion module, can realize the efficient enhancement of ton bag goods image and multi-dimensional feature fusion, the space alignment weighted superposition of color cluster and texture cluster is generated Fusion segmentation chart, let the feature representation of ton bag goods homogeneous region be more comprehensive, accurate, effectively improve the integrity and effectiveness of geometric feature extraction, so that the feature analysis ability of system to ton bag goods image is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a robust visual inspection system for ton bag cargo driven by multi-cluster feature fusion and posture correction. Background Technology

[0002] In the field of industrial warehousing and logistics transportation, visual inspection of ton bag goods is a key link in cargo management. Existing visual inspection technologies in ton bag inspection scenarios mostly adopt a single feature extraction method for image segmentation and analysis, failing to take into account the multi-dimensional features of color and texture of ton bag goods. When faced with inspection scenarios with stacked ton bags and complex surface textures, the accuracy of image segmentation is insufficient, making it difficult to effectively identify homogeneous areas of ton bag goods, thus creating potential errors for subsequent feature extraction and status determination.

[0003] In the field of industrial warehousing and logistics transportation, visual inspection of ton bag cargo is a key link in cargo management. Existing visual inspection technologies in ton bag inspection scenarios mostly use a single feature extraction method for image segmentation and analysis, failing to take into account the multi-dimensional features of color and texture of ton bags. In scenarios where ton bags are stacked in multiple layers and the surface woven texture and printed markings are intertwined, the image segmentation accuracy is insufficient, making it difficult to effectively identify homogeneous areas of ton bags. It is easy to misjudge the overlapping areas of textures of adjacent stacked ton bags as a single area, which lays the potential for errors in subsequent feature extraction and status determination. Meanwhile, existing technologies lack targeted posture correction designs. In actual warehousing and transportation, ton bag goods often exhibit posture deviations such as tilting and rotation, with horizontal and oblique placement being particularly common. Due to the lack of a unified posture normalization process, the geometric feature extraction results have significant deviations. The misjudgment and omission rates for individual ton bag region segmentation and state determination remain high. It is easy to misjudge a complete ton bag with a tilted posture as a damaged ton bag, or to miss ton bags with oblique edges during inventory. The robustness and adaptability of the detection are poor, and it cannot meet the actual needs of industrial scenarios for efficient and accurate detection of ton bag goods. Summary of the Invention

[0004] To achieve the above objectives, this invention provides a robust visual inspection system for ton bag cargo driven by multi-cluster feature fusion and pose correction. The system includes a nonlocal mean enhancement module, a multi-feature clustering fusion module, a geometric feature extraction module, a dominant pose clustering module, a pose correction transformation module, and a cargo segmentation determination module, wherein:

[0005] The nonlocal mean enhancement module is used to perform nonlocal mean filtering on the original image of the ton bag cargo to obtain an enhanced image of the ton bag cargo;

[0006] The multi-feature clustering fusion module is used to perform multi-dimensional clustering analysis on the enhanced image, and to spatially align and weight the color clusters and texture clusters obtained after analysis to obtain a fused segmentation map of the ton bag cargo.

[0007] The geometric feature extraction module is used to extract the center point coordinates and principal axis tilt angle of homogeneous regions in the fused segmentation map to obtain the geometric feature set of the ton bag cargo;

[0008] The dominant posture clustering module is used to perform one-dimensional mean-shift clustering using the principal axis tilt angles in the geometric feature set as samples to obtain the dominant posture angles of the ton bag cargo.

[0009] The attitude correction transformation module is used to perform an affine transformation on the fused segmentation map with the dominant attitude angle as the rotation parameter and the geometric centroid of the coordinates of all center points in the geometric feature set as the rotation center, so as to obtain the attitude normalization map of the ton bag cargo.

[0010] The cargo segmentation and determination module is used to segment the posture normalization map into individual ton bag regions based on preset prior knowledge of cargo shape, and to determine the state of each individual ton bag region to obtain the detection result of the ton bag cargo.

[0011] In a preferred embodiment, the process of obtaining an enhanced image of the ton bag cargo is as follows:

[0012] The original image of the ton bag cargo is acquired, and a search window and a similarity comparison window are set on the original image according to the resolution and size of the original image. The pixels in the search window are used as the pixels to be processed.

[0013] The current comparison window is extracted from the original image, centered on the position of the pixel to be processed in the original image.

[0014] Centered on the position of the reference pixel in the search window in the original image, extract the corresponding reference comparison window from the original image respectively;

[0015] Compare the grayscale distribution similarity between the current comparison window and the reference comparison window, and assign weighting coefficients to the reference pixels based on the grayscale distribution similarity.

[0016] Based on the weighting coefficients, the gray values ​​of the reference pixels are weighted and convolved to obtain the updated gray values ​​of the pixels to be processed.

[0017] Each pixel in the original image is sequentially used as a pixel to be processed. The steps of current comparison window extraction, reference comparison window extraction, weighting coefficient assignment, and weighted convolution are repeated to obtain the updated grayscale value of each pixel. The updated grayscale values ​​of all pixels constitute the enhanced image of the ton bag cargo.

[0018] In a preferred embodiment, the process of obtaining the fused segmentation map of the ton bag cargo is as follows:

[0019] The enhanced image is converted to a preset color space to obtain the color space image of the enhanced image, and local binary mode encoding is performed on the enhanced image to obtain the texture encoded image of the enhanced image;

[0020] Clustering of color attributes of pixels in a color space image yields a color label map of the color space image;

[0021] Pixels with similar texture attributes in a texture-coded image are grouped into the same texture cluster to obtain the texture label map of the texture-coded image.

[0022] The distribution frequency of the corresponding texture labels in the texture label map is calculated for the pixels covered by the color labels in the color label map, and the texture label with the highest distribution frequency is determined as the associated texture label of the color label.

[0023] Obtain the color label of the pixel in the color label map and the texture label in the texture label map in the enhanced image. If the texture label and the associated texture label of the color label are consistent, the fusion label of the pixel is assigned as the color label. If they are inconsistent, the fusion label is assigned according to the fusion label of the majority of pixels in the neighborhood of the pixel to generate the fusion label map of the ton bag of goods.

[0024] Connected components of adjacent pixels with the same fusion label in the fusion label image are merged, and the edges of the merged connected components are smoothed to obtain the fusion segmentation image of the ton bag cargo.

[0025] In a preferred embodiment, the process of obtaining the color space image of the enhanced image is as follows:

[0026] Obtain the red, green, and blue channel values ​​of pixels in the enhanced image in the red-green-blue color space;

[0027] By performing channel linear decoupling on the red, green, and blue channel values, the chroma component, saturation component, and luminance component values ​​of the pixel are obtained.

[0028] The chroma, saturation, and luminance components are recombined into channels to obtain the color space image of the enhanced image.

[0029] In a preferred embodiment, the process of obtaining the geometric feature set of ton bag cargo is as follows:

[0030] Connectivity labeling is performed on homogeneous regions in the fused segmentation image to obtain region labeling images of homogeneous regions;

[0031] Extract the pixels corresponding to the region identifiers in the region identifier image, and perform boundary tracking on the extracted pixels to obtain the closed contour lines of the homogeneous region.

[0032] Based on the closed contour line, construct the minimum bounding rectangle of the homogeneous region, extract the coordinates of the center point of the rectangle from the minimum bounding rectangle as the coordinates of the center point of the homogeneous region, and extract the direction angle of the long side of the rectangle from the minimum bounding rectangle as the principal axis inclination angle of the homogeneous region.

[0033] The coordinates of the center points and the principal axis inclinations of all homogeneous regions are aggregated into a geometric feature set for ton bag cargo.

[0034] In a preferred embodiment, the process of obtaining the dominant attitude angle of the ton bag cargo is as follows:

[0035] Extract the principal axis tilt angles of all homogeneous regions from the geometric feature set, and arrange the principal axis tilt angles as sample points on a one-dimensional angle axis;

[0036] Centered on the sample point, an initial neighborhood interval is defined on a one-dimensional angle axis according to a preset neighborhood radius. Sample points falling within the initial neighborhood interval are collected to obtain the neighborhood sample set of the sample point.

[0037] The position corresponding to the mean angle of all sample points in the neighborhood sample set is taken as the center point of the updated neighborhood interval. The neighborhood interval is redefined with the updated center point as the center. The collection of neighborhood sample set and the updating of center point are repeated until the center point position of all neighborhood intervals remains fixed in the two updates to obtain the target cluster center of the ton bag cargo.

[0038] The number of sample points attracted by the target cluster center is counted, and the angle value corresponding to the cluster center with the largest number of sample points is determined as the dominant attitude angle of the ton bag cargo.

[0039] In a preferred embodiment, the process of obtaining the attitude normalization map of the ton bag cargo is as follows:

[0040] By calibrating the centroid position of the center point of the homogeneous region with geometric features, the rotation center point of the ton bag cargo is obtained.

[0041] Using the rotation center point as a reference and the dominant pose angle as the rotation amount, the original pixel points in the fused segmentation map are rotated and mapped to obtain the mapped coordinate positions of the original pixel points.

[0042] Assign pixel values ​​to the corresponding original pixels at the mapped coordinate positions on a blank canvas to generate an initial rotated image of the ton bag cargo.

[0043] Pixel interpolation is performed on the unassigned pixels in the initial rotated image caused by coordinate transformation to obtain the pose normalized image of the ton bag cargo.

[0044] In a preferred embodiment, the process of obtaining the rotation center point of the ton bag cargo is as follows:

[0045] Traverse the coordinates of the center points of homogeneous regions in the geometric feature set to extract the x-coordinate and y-coordinate components of the center point coordinates.

[0046] The horizontal and vertical components are summed and aggregated separately to obtain the summed values ​​of the horizontal and vertical components.

[0047] The total number of homogeneous regions in the geometric feature set is counted, and the cumulative values ​​of the horizontal and vertical coordinates are mapped to the total number to obtain the horizontal and vertical coordinates of the rotation center of the ton bag cargo. The horizontal and vertical coordinates of the rotation center together constitute the rotation center point of the ton bag cargo.

[0048] In a preferred embodiment, the process of obtaining the inspection results of the ton bag cargo is as follows:

[0049] Connectivity labeling is performed on the attitude normalization graph to obtain candidate regions of the attitude normalization graph, and each candidate region is assigned a unique identifier.

[0050] Based on the pre-set prior knowledge of cargo shape, the area and perimeter values ​​of candidate regions are quantified and filtered using geometric parameters to obtain the individual regions of ton bag cargo.

[0051] Track the number of grayscale jumps of adjacent pixels on the boundary contour of the ton bag individual region, and compare the number of grayscale jumps with the grayscale jump rules of the intact contour recorded in the prior knowledge of the cargo shape to obtain the state information of the ton bag individual region.

[0052] The unique identifier of each ton bag area, its center point coordinates, and the corresponding status information are integrated into the detection results of the ton bag cargo.

[0053] In a preferred embodiment, the process of obtaining the state information of an individual area of ​​a ton bag is as follows:

[0054] Record the grayscale value changes between every two adjacent pixels on the boundary contour of the ton bag individual region, mark the positions of adjacent pixels where the grayscale value jumps as jump points, count the total number of all jump points on the boundary contour, and obtain the number of grayscale jumps in the ton bag individual region.

[0055] Extract the grayscale transition rules of the intact outline from the prior knowledge of the cargo form. The grayscale transition rules of the intact outline include the range of standard transition number and the distribution pattern of standard transition points in the intact state.

[0056] When the number of grayscale jumps falls within the standard number of jumps and the distribution of jump points matches the standard jump point distribution pattern, the status information of the individual area of ​​the ton bag is determined to be in good condition.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. This invention is equipped with a nonlocal mean enhancement module and a multi-feature clustering fusion module, which can achieve efficient enhancement and multi-dimensional feature fusion of ton bag cargo images. By generating a fused segmentation map through spatial alignment and weighted superposition of color clusters and texture clusters, the feature representation of homogeneous regions of ton bag cargo is more comprehensive and accurate, effectively improving the integrity and effectiveness of geometric feature extraction, and significantly improving the system's feature analysis capability for ton bag cargo images.

[0059] 2. This invention achieves posture normalization processing of ton bag cargo through advantageous posture clustering and posture correction transformation, making the geometric feature analysis of ton bag cargo more uniform. Combined with pre-set cargo shape prior knowledge, it performs ton bag individual region segmentation and state determination, which can significantly improve the accuracy and robustness of ton bag individual recognition and state detection. The overall detection process of the system is improved in terms of continuity and efficiency, and it can stably output accurate ton bag cargo detection results, effectively improving the overall quality and efficiency of ton bag cargo visual inspection.

[0060] 3. This invention adopts a highly modular architecture design, in which each functional module can be implemented independently and supports flexible invocation. Without modifying the core program code, cluster-style horizontal expansion can be achieved by adding or deleting modules. It can quickly adapt to the ton bag inspection needs in different warehousing and logistics scenarios, greatly reducing the secondary development cost and later maintenance difficulty of the system. At the same time, the system can be deployed on cloud servers or various industrial hardware devices, realizing the cloud-based and distributed deployment of inspection services, improving the system's adaptability and deployment flexibility in industrial scenarios.

[0061] 4. This invention incorporates multiple refined processing techniques throughout the entire image processing workflow. Weighted convolution in the image enhancement stage ensures the preservation of image details. Edge smoothing in the feature fusion stage solves the problem of irregular segmentation edges caused by overlapping textures of ton bags. Pixel interpolation in the pose correction stage fills in pixel gaps caused by coordinate transformation. The state determination stage combines dual determination logic based on the number of grayscale jumps and the distribution pattern of jump points, overcoming the limitations of single-parameter quantitative determination. It solves the detection problems caused by ton bag stacking, pose deviation, and complex textures in industrial scenarios from multiple dimensions, significantly reducing the false positive and false negative rates in the detection process, making the detection results more objective and accurate. Attached Figure Description

[0062] Figure 1 This is a system architecture diagram of a robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and posture correction, provided in an embodiment of the present invention.

[0063] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0065] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “the” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0066] Depending on the context, the word "if" or "if" as used here can be interpreted as "when" or "when" or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination" or "in response to determination" or "when detection (of the stated condition or event)" or "in response to detection (of the stated condition or event)."

[0067] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0068] In practice, the server-side equipment deployed in a robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction may consist of one or more devices. This robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction can be implemented as: a business instance, a virtual machine, or hardware devices. For example, this robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction can be implemented as a business instance deployed on one or more devices in a cloud node. Simply put, this robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction can be understood as software deployed on a cloud node, used to provide various... The user end provides a robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction. Alternatively, this robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction can also be implemented as a virtual machine deployed on one or more devices in a cloud node, with application software installed in the virtual machine for managing each user end. Or, this robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction can also be implemented as a server consisting of numerous identical or different types of hardware devices, with one or more hardware devices configured to provide each user end with a robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction.

[0069] In terms of implementation, a robust visual inspection system for ton-bag cargo based on multi-cluster feature fusion and posture correction is adapted to the user client. Specifically, if the system is implemented as an application installed on a cloud service platform, the user client acts as a client establishing a communication connection with that application; or if the system is implemented as a website, the user client acts as a webpage; or if the system is implemented as a cloud service platform, the user client acts as a mini-program within an instant messaging application.

[0070] like Figure 1 The figure shown is a system architecture diagram of a robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and posture correction driven by an embodiment of the present invention.

[0071] The present invention discloses a robust visual inspection system 100 for ton bag cargo driven by multi-cluster feature fusion and posture correction. This system can be set up in a cloud server. In terms of implementation, it can be used as one or more service devices, or as an application installed in the cloud (such as a mobile service operator's server, server cluster, etc.), or it can be developed into a website. Depending on the functions implemented, the robust visual inspection system 100 for ton bag cargo driven by multi-cluster feature fusion and posture correction may include a nonlocal mean enhancement module 101, a multi-feature clustering fusion module 102, a geometric feature extraction module 103, a dominant posture clustering module 104, a posture correction transformation module 105, and a cargo segmentation determination module 106. The modules of the present invention can also be called units, which refer to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0072] In this embodiment of the invention, a robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction is provided. Each of the above modules can be implemented independently and can call other modules. This calling can be understood as a module connecting to multiple modules of another type and providing corresponding services to those connected modules. This embodiment of the invention provides a robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction. Without modifying the program code, the applicable scope of the system architecture can be adjusted by adding modules and directly calling them, achieving cluster-based horizontal expansion. This allows for quick and flexible expansion of the robust visual inspection system for ton bags driven by multi-cluster feature fusion and posture correction. In practical applications, the above modules can be set in the same device or different devices, or in virtual devices, such as service instances in a cloud server.

[0073] The following describes, with reference to specific embodiments, each component and specific workflow of a robust visual inspection system for ton bags driven by multi-cluster feature fusion and pose correction:

[0074] The nonlocal mean enhancement module 101 is used to perform nonlocal mean filtering on the original image of the ton bag cargo to obtain an enhanced image of the ton bag cargo.

[0075] In this embodiment of the invention, the process of obtaining an enhanced image of the ton bag cargo is as follows:

[0076] The original image of the ton bag cargo is acquired, and a search window and a similarity comparison window are set on the original image according to the resolution and size of the original image. The pixels in the search window are used as the pixels to be processed.

[0077] First, acquire the original digital image of the ton bagged goods. This type of image is presented in the form of a pixel matrix. Then, based on the resolution size composed of the horizontal and vertical pixel counts of the original image, define a rectangular search window and a rectangular similarity comparison window within the pixel matrix of the original image. The horizontal and vertical pixel counts of the similarity comparison window should be smaller than those of the search window. After the definition is completed, all pixels within the pixel range of the search window are defined as pixels to be processed.

[0078] The current comparison window is extracted from the original image, centered on the position of the pixel to be processed in the original image.

[0079] Based on the x-coordinate and y-coordinate values ​​of the pixel to be processed in the pixel matrix of the original image, its specific spatial location is determined. Then, using this location as the geometric center, a pixel region with the same pixel size as the preset similarity comparison window is extracted from the pixel matrix of the original image. This extracted pixel region is the current comparison window.

[0080] Centered on the position of the reference pixel in the search window in the original image, extract the corresponding reference comparison window from the original image respectively;

[0081] Within the defined search window pixel range, each pixel other than the pixel to be processed is sequentially defined as a reference pixel. First, the spatial position of each reference pixel is determined according to the corresponding horizontal and vertical coordinate values ​​in the pixel matrix of the original image. Then, with this spatial position of each reference pixel as the geometric center, a pixel region with the same pixel size as the preset similarity comparison window is extracted from the pixel matrix of the original image. Each extracted pixel region is the reference comparison window for the corresponding reference pixel.

[0082] Compare the grayscale distribution similarity between the current comparison window and the reference comparison window, and assign weighting coefficients to the reference pixels based on the grayscale distribution similarity.

[0083] The grayscale value of each pixel in the current comparison window and the grayscale value of each pixel in the corresponding reference comparison window are extracted one by one. According to the relative position of the pixels in the window, the grayscale values ​​of corresponding pixels in the two windows are compared in turn. Then, the number of pixels with completely identical grayscale values ​​in the two windows is counted, and the proportion of this number to the total number of pixels in a single window is calculated. This proportion is the grayscale distribution similarity between the current comparison window and the reference comparison window. Subsequently, a weighting coefficient is assigned to the corresponding reference pixel based on this proportion. The higher the proportion of grayscale distribution similarity, the larger the weighting coefficient assigned to the reference pixel. If the proportion of grayscale distribution similarity is zero, the weighting coefficient assigned to the reference pixel is also zero.

[0084] Based on the weighting coefficients, the gray values ​​of the reference pixels are weighted and convolved to obtain the updated gray values ​​of the pixels to be processed.

[0085] Extract the weighting coefficient and gray value of each reference pixel. Multiply the weighting coefficient and gray value of each reference pixel. Then sum the values ​​obtained after multiplying all reference pixels. The result of this summation is the updated gray value of the pixel to be processed.

[0086] Each pixel in the original image is taken as a pixel to be processed in turn. The steps of current comparison window extraction, reference comparison window extraction, weighting coefficient assignment and weighted convolution are repeated to obtain the updated gray value of each pixel. The updated gray values ​​of all pixels constitute the enhanced image of the ton bag cargo.

[0087] The process iterates through all pixels in the original image pixel matrix, treating each pixel as a pixel to be processed. For each pixel to be processed, the operations of current comparison window extraction, reference comparison window extraction, weighted coefficient assignment, and weighted convolution are performed sequentially to calculate the corresponding updated grayscale value for each pixel in the original image. Then, the original grayscale value of each pixel in the original image is replaced with the corresponding updated grayscale value. Finally, the pixel matrix composed of the updated grayscale values ​​of all replaced pixels is the enhanced image of the ton bag of goods.

[0088] The multi-feature clustering fusion module 102 is used to perform multi-dimensional clustering analysis on the enhanced image, and to spatially align and weight the color clusters and texture clusters obtained after analysis to obtain a fused segmentation map of the ton bag cargo.

[0089] In this embodiment of the invention, the process of obtaining the fused segmentation map of the ton bag cargo is as follows:

[0090] The enhanced image is converted to a preset color space to obtain a color space image of the enhanced image, including:

[0091] Obtain the red, green, and blue channel values ​​of pixels in the enhanced image in the red-green-blue color space;

[0092] Iterate through all pixels in the enhanced image pixel matrix one by one, and extract the independent red, green and blue channel values ​​of each pixel in the red-green-blue color space. Each pixel can correspond to a unique set of red, green and blue channel values. After all the channel values ​​of all pixels have been extracted, associate these channel values ​​with the spatial position of the corresponding pixel and save them.

[0093] By performing channel linear decoupling on the red, green, and blue channel values, the chroma component, saturation component, and luminance component values ​​of the pixel are obtained.

[0094] For each pixel, a linear decoupling operation is performed on the red, green, and blue channel values ​​associated with it. Independent linear transformations and separation processes are applied to these three channel values, separating the values ​​representing the color type into chroma components, the values ​​representing the color vividness into saturation components, and the values ​​representing the color brightness into luminance components. Each pixel can ultimately correspond to a unique set of chroma, saturation, and luminance component values.

[0095] The chroma, saturation, and luminance components are recombined into channels to obtain the color space image of the enhanced image.

[0096] The original pixel matrix spatial structure and pixel position correspondence of the enhanced image are preserved. The chroma component value, saturation component value and luminance component value obtained by decoupling each pixel are used as new channel values. The channel is recombined strictly in the channel order of chroma, saturation and luminance. Finally, the pixel matrix composed of the three component channel values ​​of all the recombined pixels is the color space image of the enhanced image.

[0097] Local binary pattern coding is performed on the enhanced image to obtain the texture-coded image of the enhanced image;

[0098] A 3×3 square neighborhood window is defined centered on each pixel in the enhanced image. First, the gray value of the center pixel within the window is extracted as the reference gray value. Then, the gray values ​​of the remaining pixels within the window are compared with this reference gray value. Pixels with gray values ​​higher than the reference gray value are directly assigned a value of 1, and pixels with gray values ​​lower than the reference gray value are directly assigned a value of 0. Next, the 0s and 1s assigned in the window are arranged in a clockwise direction to form a binary number string. This number string is the local binary pattern encoding value of the center pixel. After calculating the corresponding local binary pattern encoding values ​​for all pixels in the enhanced image, these encoding values ​​are arranged according to the original pixel matrix positions. The resulting image is the texture-coded image of the enhanced image.

[0099] Clustering of color attributes of pixels in a color space image yields a color label map of the color space image;

[0100] Extract the chroma and saturation component values ​​of all pixels in the color space image, and use these two values ​​as the color attribute features of the pixels. Pixels with identical chroma and saturation component values ​​are grouped into the same cluster group, and each cluster group is assigned a unique color label. Then, the color label of each pixel corresponding to the cluster group is assigned to that pixel. The spatial positional relationship of the original pixel matrix of the image is preserved. Finally, the image composed of the color labels of all pixels arranged in order of position is the color label map of the color space image.

[0101] Pixels with similar texture attributes in a texture-coded image are grouped into the same texture cluster to obtain the texture label map of the texture-coded image.

[0102] The local binary pattern encoding value of each pixel in the texture-coded image is used as the texture attribute feature of that pixel. Pixels with the same encoding value are grouped into the same texture cluster. Each texture cluster is assigned a unique texture label. Then, the texture label of the texture cluster corresponding to each pixel is assigned to that pixel. The spatial positional relationship of the original pixel matrix of the image is preserved. Finally, the image composed of the texture labels of all pixels arranged in position is the texture label map of the texture-coded image.

[0103] The distribution frequency of the corresponding texture labels in the texture label map is calculated for the pixels covered by the color labels in the color label map, and the texture label with the highest distribution frequency is determined as the associated texture label of the color label.

[0104] Iterate through each unique color label in the color label map, first extracting the spatial location information of all pixels covered by the color label, then finding the corresponding pixel in the texture label map based on this location information and extracting its texture label. Next, count all the extracted texture labels, accurately recording the specific frequency of each texture label, and determining the texture label with the highest frequency value as the associated texture label of the color label. Match all unique color labels in the color label map with their corresponding associated texture labels in this way.

[0105] Obtain the color label of the pixel in the color label map and the texture label in the texture label map in the enhanced image. If the texture label and the associated texture label of the color label are consistent, the fusion label of the pixel is assigned as the color label. If they are inconsistent, the fusion label is assigned according to the fusion label of the majority of pixels in the neighborhood of the pixel to generate the fusion label map of the ton bag of goods.

[0106] For each pixel in the enhanced image, based on its spatial location, the corresponding color label and texture label are extracted from the color label map and texture label map respectively. First, it is determined whether the extracted texture label is exactly the same as the associated texture label corresponding to the color label. If the result is yes, the fusion label of the pixel is directly assigned to its corresponding color label. If the result is no, a 3×3 square neighborhood window is drawn with the pixel as the center, and the fusion labels of the pixels that have completed the fusion label assignment within the window are extracted. The specific number of each fusion label appearing in the window is counted, and the fusion label with the most appearances is assigned to the pixel. After all pixels in the enhanced image have completed the fusion label assignment, the spatial positional relationship of the original pixel matrix of the image is preserved. The image composed of the fusion labels of all pixels arranged in position is the fusion label map of the ton bag cargo.

[0107] Connected components of adjacent pixels with the same fusion label in the fusion label image are merged, and the edges of the merged connected components are smoothed to obtain the fusion segmentation image of the ton bag cargo.

[0108] Using the four-neighbor connectivity rule as the criterion, all pixels in the fused label image are traversed. Pixels that are adjacent in the horizontal and vertical directions and have the same fusion label are grouped into the same connected region. The identification and merging of all connected regions are completed. Then, the edge pixels of each merged connected region are traversed one by one. Irregular pixels with uneven edges are replaced with the average label attribute of the pixels in the same connected region in its neighborhood. This eliminates all irregular pixels at the edges of the connected regions. The image formed after connected region merging and edge smoothing is the fused segmentation image of the ton bag cargo.

[0109] The geometric feature extraction module 103 is used to extract the center point coordinates and principal axis tilt angle of homogeneous regions in the fused segmentation map to obtain the geometric feature set of the ton bag cargo;

[0110] In this embodiment of the invention, the process of obtaining the geometric feature set of ton bag cargo is as follows:

[0111] Connectivity labeling is performed on homogeneous regions in the fused segmentation image to obtain region labeling images of homogeneous regions;

[0112] The pixel matrix of the fused segmentation image is traversed one by one. Pixels are identified according to the eight-neighbor connectivity rule. All pixels that are interconnected and belong to the same homogeneous region are grouped together. Each independent homogeneous region is assigned a unique numerical identifier. This numerical identifier is then assigned to each pixel in the corresponding homogeneous region. Background pixels that do not belong to any homogeneous region are uniformly assigned a zero identifier. The original spatial position relationship of the pixel matrix of the fused segmentation image is preserved. Finally, the pixel matrix formed by arranging the numerical identifiers of all pixels in their original positions is the region identifier image of the homogeneous region.

[0113] Extract the pixels corresponding to the region identifiers in the region identifier image, and perform boundary tracking on the extracted pixels to obtain the closed contour lines of the homogeneous region.

[0114] The pixel matrix of the region identifier image is traversed one by one to extract all pixels with non-zero identifiers. These pixels are the pixels corresponding to the region identifiers. Then, the pixels are sorted according to their numerical identifiers to form a set of pixels corresponding to each homogeneous region. For each set of pixels, boundary tracking is performed starting from the pixel with the smallest horizontal coordinate and the smallest vertical coordinate. Adjacent region identifier pixels are searched in a clockwise direction, and the spatial coordinates of each searched boundary pixel are recorded completely until the search trajectory returns to the spatial coordinates of the starting pixel. The spatial coordinates of all the boundary pixels recorded in sequence are connected to form a continuous line, which is the closed contour line of the homogeneous region. The corresponding closed contour line is generated for each independent homogeneous region in this way.

[0115] Based on the closed contour line, construct the minimum bounding rectangle of the homogeneous region, extract the coordinates of the center point of the rectangle from the minimum bounding rectangle as the coordinates of the center point of the homogeneous region, and extract the direction angle of the long side of the rectangle from the minimum bounding rectangle as the principal axis inclination angle of the homogeneous region.

[0116] For each homogeneous region's closed contour line, extract the x and y coordinates of all pixels on the contour line. Then, determine the maximum and minimum values ​​of the x and y coordinates respectively. Use the minimum x and y coordinates as the coordinates of the lower left corner of the rectangle, and the maximum x and y coordinates as the coordinates of the upper right corner of the rectangle. Construct a rectangle that fits and completely encloses the closed contour line. This rectangle is the minimum bounding rectangle of the homogeneous region. Calculate the x and y coordinates of the horizontal midpoint and vertical midpoint of the minimum bounding rectangle. The coordinates formed by these two values ​​are the coordinates of the rectangle's center point. Directly use these coordinates as the center point coordinates of the homogeneous region. Next, identify the direction of the long side of the minimum bounding rectangle and calculate the angle between this long side direction and the horizontal rightward direction. Use this angle as the long side direction angle of the rectangle, and also directly use this angle as the principal axis tilt angle of the homogeneous region. Extract the corresponding center point coordinates and principal axis tilt angle for each independent homogeneous region using this method.

[0117] The coordinates of the center points and the principal axis inclinations of all homogeneous regions are combined into a geometric feature set of ton bag cargo;

[0118] First, establish a feature storage set. Create a unique feature entry for each independent homogeneous region. Accurately record the center point coordinates and principal axis tilt angle of each homogeneous region into its exclusive feature entry. Then, arrange all feature entries in the numerical order of the region identifiers. After the center point coordinates and principal axis tilt angles of all homogeneous regions have been recorded, this feature storage set containing all feature entries is the geometric feature set of the ton bag cargo.

[0119] The dominant posture clustering module 104 is used to perform one-dimensional mean-shift clustering with the principal axis tilt angles in the geometric feature set as samples to obtain the dominant posture angles of the ton bag cargo.

[0120] In this embodiment of the invention, the process of obtaining the dominant attitude angle of the ton bag cargo is as follows:

[0121] Extract the principal axis tilt angles of all homogeneous regions from the geometric feature set, and arrange the principal axis tilt angles as sample points on a one-dimensional angle axis;

[0122] The process iterates through all feature entries of homogeneous regions in the geometric feature set of ton bag cargo, extracting the corresponding principal axis tilt angle value from each feature entry. First, a one-dimensional angle axis with a value range of 0 to 360 degrees is set. Then, the angle value of each principal axis tilt angle is marked on the corresponding position of the one-dimensional angle axis according to the actual value. Each angle value corresponds to an independent sample point on the one-dimensional angle axis. After all the principal axis tilt angles of homogeneous regions are marked, a whole of sample points composed of all principal axis tilt angles is formed on the one-dimensional angle axis.

[0123] Centered on the sample point, an initial neighborhood interval is defined on a one-dimensional angle axis according to a preset neighborhood radius. Sample points falling within the initial neighborhood interval are collected to obtain the neighborhood sample set of the sample point.

[0124] Taking each sample point on the one-dimensional angle axis as the center, a preset fixed angle range is used as the neighborhood radius. Based on the angle value corresponding to the sample point, the same angle value is extended to the left and right sides to determine the left and right critical values ​​of the angle interval. The angle range between the two critical values ​​is the initial neighborhood interval of this sample point. Then, the initial neighborhood interval of each sample point is searched one by one, and all sample points whose angle values ​​fall within this interval are selected and collected. Then, these selected and collected sample points are integrated into a dedicated set. This set is the neighborhood sample set of the corresponding sample point. In this way, a corresponding neighborhood sample set is generated for all sample points on the one-dimensional angle axis.

[0125] The position corresponding to the mean angle of all sample points in the neighborhood sample set is taken as the center point of the updated neighborhood interval. The neighborhood interval is redefined with the updated center point as the center. The collection of neighborhood sample set and the updating of center point are repeated until the center point position of all neighborhood intervals remains fixed in the two updates to obtain the target cluster center of the ton bag cargo.

[0126] For each sample point, the angle values ​​of all sample points in the neighborhood sample set are extracted and summed. The summed value is divided by the actual total number of sample points in the neighborhood sample set. The calculated value is the mean angle of the neighborhood sample set. The position corresponding to this mean angle on the one-dimensional angle axis is found and set as the center point of the updated neighborhood interval. Then, based on the updated center point, the angle interval is redefined according to the same neighborhood radius as the initial neighborhood interval. Sample points in the new angle interval are retrieved to form a new neighborhood sample set. The mean angle is recalculated to update the center point. This process of collecting neighborhood sample sets and updating center points is repeated until the angle value of the updated center point is exactly the same as the angle value of the previously updated center point. At this point, the iteration operation of this neighborhood interval is stopped. After all neighborhood intervals have been iterated and updated, all the center points that remain fixed are determined as the target cluster centers of the ton bag cargo.

[0127] The number of sample points attracted by the target cluster center is counted, and the angle value corresponding to the cluster center with the largest number of sample points is determined as the dominant attitude angle of the ton bag cargo.

[0128] For each target cluster center, a stable neighborhood interval is retrieved, and the actual number of sample points falling within this interval is counted. This number represents the number of sample points attracted by the target cluster center. The number of attracted sample points for each target cluster center is compared one by one, and the target cluster center with the largest number is identified. The angle value corresponding to this target cluster center on the one-dimensional angle axis is extracted, and this angle value is directly determined as the dominant attitude angle of the ton bag cargo. If multiple target cluster centers have the same number of attracted sample points and all of them are the maximum value, the average of the angle values ​​corresponding to these cluster centers is taken as the dominant attitude angle of the ton bag cargo, thus ensuring the uniqueness of the dominant attitude angle.

[0129] The attitude correction transformation module 105 is used to perform an affine transformation on the fused segmentation map with the dominant attitude angle as the rotation parameter and the geometric centroid of the coordinates of all center points in the geometric feature set as the rotation center, so as to obtain the attitude normalization map of the ton bag cargo.

[0130] In this embodiment of the invention, the process of obtaining the attitude normalization diagram of the ton bag cargo is as follows:

[0131] By calibrating the centroid position of the center point coordinates of the homogeneous region in the geometric feature set, the rotation center point of the ton bag cargo is obtained, including:

[0132] Traverse the coordinates of the center points of homogeneous regions in the geometric feature set to extract the x-coordinate and y-coordinate components of the center point coordinates.

[0133] Iterate through the center point coordinate entries of all homogeneous regions in the geometric feature set one by one, and separate each center point coordinate into independent horizontal and vertical coordinate components. Arrange all the extracted horizontal coordinate components into a horizontal coordinate component set in the extraction order, and arrange the vertical coordinate components into a vertical coordinate component set in the same order to ensure that the horizontal and vertical coordinate components of each homogeneous region can correspond one-to-one.

[0134] The horizontal and vertical components are summed and aggregated separately to obtain the summed values ​​of the horizontal and vertical components.

[0135] The sum of all values ​​in the x-axis component set is accumulated one by one until all components in the set are added. The final sum is the accumulated value of the x-axis. The same method is used to accumulate all values ​​in the y-axis component set one by one. The final sum is the accumulated value of the y-axis. The entire accumulation process retains the original values ​​of all components without any deletion or selection.

[0136] The total number of homogeneous regions in the geometric feature set is counted, and the cumulative values ​​of the horizontal and vertical coordinates are mapped to the total number to obtain the horizontal and vertical coordinates of the rotation center of the ton bag cargo. The horizontal and vertical coordinates of the rotation center together constitute the rotation center point of the ton bag cargo.

[0137] Count all the feature entries of homogeneous regions recorded in the geometric feature set one by one. The specific number counted is the total number of homogeneous regions in the geometric feature set. Divide the accumulated value of the horizontal coordinate by this total number. The quotient is the horizontal coordinate of the rotation center of the ton bag cargo. Then divide the accumulated value of the vertical coordinate by the total number. The quotient is the vertical coordinate of the rotation center of the ton bag cargo. Combine these two quotients into a two-dimensional coordinate form. This two-dimensional coordinate is the rotation center point of the ton bag cargo.

[0138] Using the rotation center point as a reference and the dominant pose angle as the rotation amount, the original pixel points in the fused segmentation map are rotated and mapped to obtain the mapped coordinate positions of the original pixel points.

[0139] Retrieve the previously determined rotation center point and dominant attitude angle of the ton bag cargo. Iterate through each original pixel in the pixel matrix of the fused segmentation image, extract the original two-dimensional coordinates of each original pixel, and use the rotation center point as the coordinate reference point. Rotate the original two-dimensional coordinates of the original pixel around the reference point according to the angle value of the dominant attitude angle. Determine the corresponding two-dimensional coordinates of the original pixel in the new coordinate space after rotation through position mapping. This rotated two-dimensional coordinate is the mapped coordinate position of the original pixel, thus matching the corresponding mapped coordinate position of all original pixels in the fused segmentation image.

[0140] Assign pixel values ​​to the corresponding original pixels at the mapped coordinate positions on a blank canvas to generate an initial rotated image of the ton bag cargo.

[0141] Create a blank canvas with the exact same pixel matrix size as the fused segmentation image. The horizontal and vertical coordinate ranges of this blank canvas are the same as those of the fused segmentation image. Based on the mapped coordinate position of each original pixel in the fused segmentation image, accurately assign the pixel value of the original pixel to the corresponding mapped coordinate position of the pixel in the blank canvas. After the pixel value assignment operation of all original pixels is completed, the image formed by this canvas with the assigned pixels is the initial rotated image of the ton bag of goods.

[0142] Pixel interpolation is performed on the unassigned pixels in the initial rotated image caused by coordinate transformation to obtain the pose normalized image of the ton bag cargo.

[0143] The entire pixel matrix of the initial rotated image is traversed one by one, and pixels without assigned pixel values ​​are searched point by point in the canvas. These pixels are defined as unassigned pixels caused by coordinate transformation. For each unassigned pixel, the pixel values ​​of all assigned pixels in its surrounding 3×3 square neighborhood are extracted. The pixel values ​​of all assigned pixels are summed up and then divided by the actual number of assigned pixels in the neighborhood. The result is the interpolated pixel value. The calculated interpolated pixel value is assigned to the corresponding unassigned pixel. After all unassigned pixels in the initial rotated image have been assigned interpolated pixel values, the final complete image is the pose normalized image of the ton bag cargo.

[0144] The cargo segmentation and determination module 106 is used to segment the posture normalization map into individual regions of ton bags based on preset prior knowledge of cargo form, and to determine the state of the individual regions of ton bags to obtain the detection result of the ton bag cargo.

[0145] In this embodiment of the invention, the process of obtaining the detection results of the ton bag cargo is as follows:

[0146] Connectivity labeling is performed on the attitude normalization graph to obtain candidate regions of the attitude normalization graph, and each candidate region is assigned a unique identifier.

[0147] The pixel matrix of the pose normalization map is traversed one by one. According to the eight-neighbor connectivity rule, the sets of interconnected pixels in the image are identified. Each independent set of connected pixels is defined as a candidate region of the pose normalization map. Each candidate region is assigned a unique numerical identifier. This identifier is assigned to all pixels in the corresponding candidate region. Background pixels that do not belong to any candidate region are uniformly assigned a zero identifier. After all candidate regions have been assigned identifiers, the candidate regions of the pose normalization map with unique identifiers are obtained.

[0148] Based on the pre-set prior knowledge of cargo shape, the area and perimeter values ​​of candidate regions are quantified and filtered using geometric parameters to obtain the individual regions of ton bag cargo.

[0149] The system retrieves pre-defined prior knowledge of cargo morphology, which stores the standard area and perimeter ranges for ton bag cargo. First, it calculates the area of ​​each candidate region and counts the total number of pixels within that region. This count represents the area. Then, it calculates the perimeter of each candidate region and counts the total number of pixels along its boundary. This count represents the perimeter. The system compares the area and perimeter of each candidate region to the standard area and perimeter ranges. Candidate regions that simultaneously satisfy both the area and perimeter ranges are identified as the individual ton bag cargo regions.

[0150] The number of grayscale jumps of adjacent pixels on the boundary contour of a ton bag individual region is tracked, and the consistency of the grayscale jump counts is compared with the grayscale jump rules of the intact contour recorded in the prior knowledge of the cargo shape to obtain the state information of the ton bag individual region, including:

[0151] Record the grayscale value changes between every two adjacent pixels on the boundary contour of the ton bag individual region, mark the positions of adjacent pixels where the grayscale value jumps as jump points, count the total number of all jump points on the boundary contour, and obtain the number of grayscale jumps in the ton bag individual region.

[0152] Extract the closed boundary contour of each ton bag individual region, and traverse all pixels on the contour one by one in a clockwise direction. Record the gray values ​​of each pair of adjacent pixels one by one, and mark the positions where the gray values ​​of adjacent two pixels are different as transition points. After traversing the entire boundary contour and marking all transition points, count all the marked transition points one by one. The specific number counted is the number of gray-level transitions of the ton bag individual region.

[0153] Extract the grayscale transition rules of the intact outline from the prior knowledge of the cargo form. The grayscale transition rules of the intact outline include the range of standard transition number and the distribution pattern of standard transition points in the intact state.

[0154] The grayscale transition pattern of the intact outline is directly retrieved from the pre-defined prior knowledge of the cargo's form. This pattern contains two core parts: the first part is the standard number range of grayscale transitions corresponding to the intact state of the ton bag cargo, which is a fixed continuous numerical range; the second part is the standard transition point distribution pattern corresponding to the intact state of the ton bag cargo, which is a fixed combination of the interval distribution characteristics and position distribution characteristics of the transition points on the boundary outline of the individual area of ​​the ton bag. By completely extracting this numerical range and distribution feature combination, the grayscale transition pattern of the intact outline is obtained.

[0155] When the number of grayscale jumps falls within the standard number of jumps and the distribution of jump points matches the standard jump point distribution pattern, the status information of the individual area of ​​the ton bag is determined to be in good condition.

[0156] The grayscale jump count of the individual ton bag area is matched and verified with the standard jump count range in the grayscale jump pattern of the intact outline to see if the grayscale jump count falls within this fixed numerical range. At the same time, the actual distribution position of the jump points on the boundary outline of the individual ton bag area is compared with the interval and position characteristics of the standard jump point distribution pattern to see if the actual distribution completely matches the pattern characteristics. If the grayscale jump count falls within the standard jump count range and the distribution position of the jump points matches the standard jump point distribution pattern, the state information of this individual ton bag area is determined to be in an intact state.

[0157] The unique identifier of each ton bag area, its center point coordinates, and the corresponding status information are integrated into the detection results of the ton bag cargo.

[0158] Extract the unique identifier corresponding to each individual area of ​​the ton bag, construct the minimum bounding rectangle for each individual area of ​​the ton bag, and then calculate the two-dimensional coordinates of the center point of the rectangle. These coordinates are used as the center point coordinates of the individual area of ​​the ton bag. Create an independent information entry for each individual area of ​​the ton bag, and enter the unique identifier, center point coordinates, and determined status information of the area into the corresponding information entry one by one. After all the information of the individual areas of the ton bags has been entered, all the information entries are sorted and merged according to the numerical order of the unique identifier. The final complete information set is the detection result of the ton bag cargo.

[0159] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0160] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction, characterized in that, The system includes a nonlocal mean enhancement module, a multi-feature clustering fusion module, a geometric feature extraction module, a dominant pose clustering module, a pose correction and transformation module, and a cargo segmentation determination module, among which: The nonlocal mean enhancement module performs nonlocal mean filtering on the original image of the ton bag cargo to obtain an enhanced image of the ton bag cargo; The multi-feature clustering fusion module performs multi-dimensional clustering analysis on the enhanced image and spatially aligns and weights the color clusters and texture clusters obtained after the analysis to obtain a fused segmentation map of the ton bag cargo. The geometric feature extraction module extracts the center point coordinates and principal axis tilt angles of homogeneous regions in the fused segmentation map to obtain the geometric feature set of the ton bag cargo; The dominant posture clustering module uses the principal axis tilt angles in the geometric feature set as samples for one-dimensional mean-shift clustering to obtain the dominant posture angles of the ton bag cargo. The attitude correction and transformation module uses the dominant attitude angle as the rotation parameter and the geometric centroid of the coordinates of all center points in the geometric feature set as the rotation center to perform an affine transformation on the fused segmentation map to obtain the attitude normalization map of the ton bag cargo. The cargo segmentation and determination module, based on preset prior knowledge of cargo form, segments the posture normalization map into individual ton bag regions, and performs state determination on the individual ton bag regions to obtain the detection results of the ton bag cargo.

2. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 1, characterized in that, The process of obtaining an enhanced image of ton bagged goods is as follows: The original image of the ton bag cargo is acquired, and a search window and a similarity comparison window are set on the original image according to the resolution and size of the original image. The pixels in the search window are used as the pixels to be processed. The current comparison window is extracted from the original image, centered on the position of the pixel to be processed in the original image. Centered on the position of the reference pixel in the search window in the original image, extract the corresponding reference comparison window from the original image respectively; Compare the grayscale distribution similarity between the current comparison window and the reference comparison window, and assign weighting coefficients to the reference pixels based on the grayscale distribution similarity. Based on the weighting coefficients, the gray values ​​of the reference pixels are weighted and convolved to obtain the updated gray values ​​of the pixels to be processed. Each pixel in the original image is sequentially used as a pixel to be processed. The steps of current comparison window extraction, reference comparison window extraction, weighting coefficient assignment, and weighted convolution are repeated to obtain the updated grayscale value of each pixel. The updated grayscale values ​​of all pixels constitute the enhanced image of the ton bag cargo.

3. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 1, characterized in that, The process of obtaining the merged segmentation diagram of the ton bag cargo is as follows: The enhanced image is converted to a preset color space to obtain the color space image of the enhanced image, and local binary mode encoding is performed on the enhanced image to obtain the texture encoded image of the enhanced image; Clustering of color attributes of pixels in a color space image yields a color label map of the color space image; Pixels with similar texture attributes in a texture-coded image are grouped into the same texture cluster to obtain the texture label map of the texture-coded image. The distribution frequency of the corresponding texture labels in the texture label map is calculated for the pixels covered by the color labels in the color label map, and the texture label with the highest distribution frequency is determined as the associated texture label of the color label. Obtain the color label of the pixel in the color label map and the texture label in the texture label map in the enhanced image. If the texture label and the associated texture label of the color label are consistent, the fusion label of the pixel is assigned as the color label. If they are inconsistent, the fusion label is assigned according to the fusion label of the majority of pixels in the neighborhood of the pixel to generate the fusion label map of the ton bag of goods. Connected components of adjacent pixels with the same fusion label in the fusion label image are merged, and the edges of the merged connected components are smoothed to obtain the fusion segmentation image of the ton bag cargo.

4. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 3, characterized in that, The process of obtaining the color space image of the enhanced image is as follows: Obtain the red, green, and blue channel values ​​of pixels in the enhanced image in the red-green-blue color space; By performing channel linear decoupling on the red, green, and blue channel values, the chroma component, saturation component, and luminance component values ​​of the pixel are obtained. The chroma, saturation, and luminance components are recombined into channels to obtain the color space image of the enhanced image.

5. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 1, characterized in that, The process of obtaining the geometric feature set of ton bag cargo is as follows: Connectivity labeling is performed on homogeneous regions in the fused segmentation image to obtain region labeling images of homogeneous regions; Extract the pixels corresponding to the region identifiers in the region identifier image, and perform boundary tracking on the extracted pixels to obtain the closed contour lines of the homogeneous region. Based on the closed contour line, construct the minimum bounding rectangle of the homogeneous region, extract the coordinates of the center point of the rectangle from the minimum bounding rectangle as the coordinates of the center point of the homogeneous region, and extract the direction angle of the long side of the rectangle from the minimum bounding rectangle as the principal axis inclination angle of the homogeneous region. The coordinates of the center points and the principal axis inclinations of all homogeneous regions are aggregated into a geometric feature set for ton bag cargo.

6. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 1, characterized in that, The process of obtaining the dominant attitude angle of the ton bag cargo is as follows: Extract the principal axis tilt angles of all homogeneous regions from the geometric feature set, and arrange the principal axis tilt angles as sample points on a one-dimensional angle axis; Centered on the sample point, an initial neighborhood interval is defined on a one-dimensional angle axis according to a preset neighborhood radius. Sample points falling within the initial neighborhood interval are collected to obtain the neighborhood sample set of the sample point. The position corresponding to the mean angle of all sample points in the neighborhood sample set is taken as the center point of the updated neighborhood interval. The neighborhood interval is redefined with the updated center point as the center. The collection of neighborhood sample set and the updating of center point are repeated until the center point position of all neighborhood intervals remains fixed in the two updates to obtain the target cluster center of the ton bag cargo. The number of sample points attracted by the target cluster center is counted, and the angle value corresponding to the cluster center with the largest number of sample points is determined as the dominant attitude angle of the ton bag cargo.

7. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 1, characterized in that, The process of obtaining the attitude normalized diagram of the ton bag cargo is as follows: By calibrating the centroid position of the center point of the homogeneous region with geometric features, the rotation center point of the ton bag cargo is obtained. Using the rotation center point as a reference and the dominant pose angle as the rotation amount, the original pixel points in the fused segmentation map are rotated and mapped to obtain the mapped coordinate positions of the original pixel points. Assign pixel values ​​to the corresponding original pixels at the mapped coordinate positions on a blank canvas to generate an initial rotated image of the ton bag cargo. Pixel interpolation is performed on the unassigned pixels in the initial rotated image caused by coordinate transformation to obtain the pose normalized image of the ton bag cargo.

8. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 7, characterized in that, The process of obtaining the rotation center point of the ton bag cargo is as follows: Traverse the coordinates of the center points of homogeneous regions in the geometric feature set to extract the x-coordinate and y-coordinate components of the center point coordinates. The horizontal and vertical components are summed and aggregated separately to obtain the summed values ​​of the horizontal and vertical components. The total number of homogeneous regions in the geometric feature set is counted, and the cumulative values ​​of the horizontal and vertical coordinates are mapped to the total number to obtain the horizontal and vertical coordinates of the rotation center of the ton bag cargo. The horizontal and vertical coordinates of the rotation center together constitute the rotation center point of the ton bag cargo.

9. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 1, characterized in that, The process of obtaining the inspection results for ton bagged goods is as follows: Connectivity labeling is performed on the attitude normalization graph to obtain candidate regions of the attitude normalization graph, and each candidate region is assigned a unique identifier. Based on the pre-set prior knowledge of cargo shape, the area and perimeter values ​​of candidate regions are quantified and filtered using geometric parameters to obtain the individual regions of ton bag cargo. Track the number of grayscale jumps of adjacent pixels on the boundary contour of the ton bag individual region, and compare the number of grayscale jumps with the grayscale jump rules of the intact contour recorded in the prior knowledge of the cargo shape to obtain the state information of the ton bag individual region. The unique identifier of each ton bag area, its center point coordinates, and the corresponding status information are integrated into the detection results of the ton bag cargo.

10. The robust visual inspection system for ton bag cargo based on multi-cluster feature fusion and pose correction as described in claim 9, characterized in that, The process of obtaining the state information of an individual area of ​​a ton bag is as follows: Record the grayscale value changes between every two adjacent pixels on the boundary contour of the ton bag individual region, mark the positions of adjacent pixels where the grayscale value jumps as jump points, count the total number of all jump points on the boundary contour, and obtain the number of grayscale jumps in the ton bag individual region. Extract the grayscale transition rules of the intact outline from the prior knowledge of the cargo form. The grayscale transition rules of the intact outline include the range of standard transition number and the distribution pattern of standard transition points in the intact state. When the number of grayscale jumps falls within the standard number of jumps and the distribution of jump points matches the standard jump point distribution pattern, the status information of the individual area of ​​the ton bag is determined to be in good condition.

Images