Computer vision-based empty barrel automatic counting and warehousing verification method and system

Abstract

This invention discloses a computer vision-based method and system for automatic empty can counting and warehousing verification. By specifically addressing the image reflection interference problem in the complex environment of delivery stations, this invention performs reflective region segmentation and compensation processing on the images of empty cans, effectively eliminating the negative impact of reflections on the smooth surface of the cans on contour recognition. Furthermore, adaptive edge enhancement is applied to the de-reflected images, significantly improving the extraction accuracy of edge features for each empty can. Then, while accurately identifying the edge images of each empty can, brand label classification is completed, allowing for the determination of the quantity of empty cans from different brands at once, which can then be directly compared and verified with SaaS deposit invoices. Thus, this invention completely replaces the inefficient traditional method of manual counting, identification, and data entry, significantly improving the efficiency and accuracy of empty can warehousing and achieving automation and intelligence in the empty can warehousing process.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method and system for automatic empty bucket counting and warehousing verification based on computer vision. Background Technology

[0002] Bottled water, a common form of drinking water supply in daily life, involves multiple stages in its circulation management, including manufacturers, distribution stations, and end users. Among these, the counting and verification of empty bottles in the bottled water recycling system is a crucial daily operation, directly impacting deposit settlement, inventory management, and logistics efficiency. Currently, most delivery stations still rely on a manual verification model when handling returned empty bottles. Delivery personnel visually count each empty bottle, identify the brand label on the bottle, and then manually enter the count into the system for comparison with the SaaS deposit invoice.

[0003] However, the aforementioned manual method has many drawbacks. First, manual counting is easily affected by subjective factors. In a high-intensity, fast-paced work environment, errors and omissions due to fatigue and lack of concentration are common, making it difficult to guarantee accuracy. Second, manually identifying and counting the labels on different brands of containers further increases the complexity of the work and the probability of errors. Finally, the entire process is time-consuming and laborious, severely restricting the operational efficiency of the site and becoming a bottleneck in the business process.

[0004] Currently, with the rapid development of computer vision and deep learning technologies, it has become possible to achieve target detection and counting using image recognition technology. However, directly applying existing technologies to empty bucket counting scenarios faces a series of technical challenges. Among these challenges, the complex environment of delivery stations often results in strong reflections from the smooth surfaces of empty buckets, severely interfering with the clarity of the bucket outlines and negatively impacting subsequent edge detection and segmentation. Therefore, eliminating or compensating for reflective areas in the images has become the primary challenge in ensuring recognition accuracy. Furthermore, during image acquisition and transmission, indistinct edges may exist in the images, easily interfering with edge contour extraction and further reducing the accuracy of edge detection. Therefore, based on the aforementioned shortcomings, providing a method for efficient and accurate automatic counting and verification of empty buckets has become a pressing technical problem in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a computer vision-based method and system for automatic counting and warehousing verification of empty buckets, in order to solve the problems of low edge detection accuracy caused by reflections and unclear image edges in the captured images of empty buckets in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, a computer vision-based method for automatic empty bucket counting and inbound verification is provided, including: Capture images of empty buckets inside the delivery station; The empty bucket image is processed by region segmentation to separate the reflective area and the non-reflective area of ​​the bucket body in the empty bucket image. Based on the reflective area and the non-reflective area of ​​the barrel, the reflection compensation amount of each reflective pixel in the reflective area is calculated. Based on the reflection compensation amount of each reflective pixel, reflection compensation processing is performed on each reflective pixel in the empty bucket image to obtain a de-reflective image. Adaptive edge enhancement processing is performed on the de-reflected image to obtain an enhanced captured image; Edge segmentation is performed on the enhanced images to obtain the edge images of each empty bucket in the empty bucket images; Brand label recognition is performed on each edge image to obtain the brand information of each empty bucket; Based on the edge image and brand information of each empty bucket, the quantity of empty buckets of different brands in the delivery station is determined, so as to verify the empty buckets upon entry into the warehouse according to the quantity of empty buckets of different brands.

[0007] Based on the aforementioned disclosure, this invention segments the reflective areas of an empty bucket image and calculates the reflection compensation amount for each reflective pixel using both non-reflective and reflective areas. Then, based on this compensation amount, targeted compensation is applied to the reflective areas, effectively eliminating reflection interference and significantly improving image quality, laying a solid foundation for subsequent accurate edge segmentation. Furthermore, addressing the edge blurring issue that may occur during image acquisition and transmission, this invention further performs adaptive edge enhancement processing on top of the reflection removal. This operation effectively highlights the edge features of the bucket, resulting in improved edge segmentation results. More precise and complete, thus significantly improving the accuracy of recognizing the outline of each empty bucket; finally, while accurately extracting the edge image of each empty bucket, this invention, combined with brand label recognition, can complete the quantity count and brand classification of empty buckets in one go, thereby determining the quantity of empty buckets of different brands, and then directly comparing and verifying them with SaaS deposit bills. This process completely replaces the inefficient traditional mode of relying on manual counting, identification and entry, realizing the automation and intelligence of the empty bucket warehousing process, and solving the business bottleneck problem that has long restricted the efficiency of site operation; therefore, this invention is very suitable for large-scale application and promotion.

[0008] In one possible design, the image of the empty bucket is processed by region segmentation to separate the reflective areas and the non-reflective areas of the bucket body in the image, including: Calculate the chromaticity mapping vector of each pixel in the empty barrel image, and use the chromaticity mapping vector of each pixel to map the RGB values ​​of each pixel in the empty barrel image to the chromaticity unit sphere to obtain the mapped image. The mapped image is subjected to an exponential transformation to obtain the transformed image; Clustering is performed on each pixel in the transformed image to obtain two pixel clusters; Obtain the light source chromaticity vector and calculate the Euclidean distance between the cluster center of two pixel clusters and the light source chromaticity vector; Based on the Euclidean distance between the cluster centers of the two pixel clusters and the chromaticity vector of the light source, the two pixel clusters are divided into a reflective cluster and a non-reflective cluster. Based on the reflective cluster and the non-reflective cluster, the reflective area and the non-reflective area of ​​the barrel body are segmented from the image of the empty barrel.

[0009] In one possible design, based on the reflective area and the non-reflective area of ​​the barrel, the reflectivity compensation amount for each reflective pixel in the reflective area is calculated, including: For any reflective pixel in the reflective area, the RGB value of the reflective pixel is mapped onto a chromaticity unit sphere to obtain the chromaticity mapping vector of the reflective pixel. Based on the chromaticity mapping vector, the chromaticity compensation parameters of any reflective pixel are calculated. Using the chromaticity compensation parameter and chromaticity mapping vector, the first reflection suppression amount of any reflective pixel is calculated, and after traversing all reflective pixels in the reflective area, the first reflection suppression amount of each reflective pixel is obtained. Based on the non-reflective area of ​​the barrel body, the average brightness of the barrel body and the chromaticity direction component of the barrel body are calculated. The second reflectivity suppression amount is calculated using the average brightness of the barrel body and the chromaticity direction component. The reflection compensation amount for each reflective pixel in the reflective region is composed of the second reflection suppression amount and the first reflection suppression amount for each reflective pixel.

[0010] In one possible design, based on the chromaticity mapping vector, the chromaticity compensation parameters for any reflective pixel are calculated, including: Obtain the light source chromaticity vector, and based on the light source chromaticity vector and the chromaticity mapping vector of any reflective pixel, calculate the projection coefficient of the pixel color of any reflective pixel in the direction of the light source; Using the projection coefficient, the light source chromaticity vector, and the chromaticity mapping vector, the projection vector of any reflective pixel on a plane orthogonal to the light source direction is calculated; From the chromaticity mapping vector, select the maximum and minimum mapping values; Calculate the difference between the maximum and minimum mapped values, and use the ratio between the maximum and minimum mapped values ​​as the first intermediate chromaticity parameter; Calculate the magnitude of the projection vector and obtain the product between the magnitude and the difference. The ratio between the projection coefficient and the product is used as the second chromaticity intermediate parameter; Summing the first chromaticity intermediate parameter and the second chromaticity intermediate parameter yields the initial chromaticity compensation parameter; The product of the initial chromaticity compensation parameter and the difference is calculated to obtain the third intermediate chromaticity parameter; The difference between the maximum mapping value and the third chromaticity intermediate parameter is used as the chromaticity compensation parameter.

[0011] In one possible design, the chromaticity mapping vector includes chromaticity mapping values ​​for the R channel, the G channel, and the B channel. Using the chromaticity compensation parameters and the chromaticity mapping vector, the first reflection suppression amount for any reflective pixel is calculated, including: The chromaticity mapping values ​​of the R channel, G channel, and B channel are calculated, and the difference between them and the chromaticity compensation parameter is used to obtain the channel suppression amount of the R channel, G channel, and B channel, respectively. The first reflectivity suppression amount is formed by using the channel suppression amounts of the R channel, G channel, and B channel; The second reflectivity suppression amount is calculated using the average brightness and chromaticity direction components of the barrel body, including: The second reflectivity suppression amount is obtained by multiplying the average brightness of the barrel body with the chromaticity direction component. Accordingly, based on the reflection compensation amount of each reflective pixel, reflection compensation processing is performed on each reflective pixel in the empty bucket image, which includes: For any reflective pixel, the second reflection suppression amount and the first reflection suppression amount of the reflective pixel are weighted and summed to obtain the reflection-compensated pixel value of the reflective pixel. After all reflective pixels in the empty bucket image have been polled, the de-reflective image is obtained.

[0012] In one possible design, adaptive edge enhancement processing is performed on the dereflected image to obtain an enhanced captured image, including: Based on the de-reflected image, the average pixel energy of the de-reflected image is calculated; Obtain the maximum brightness value, minimum brightness value, and average brightness value of the de-reflected image; The edge enhancement adaptive threshold is calculated based on the average pixel energy, the maximum brightness value, the minimum brightness value, and the average brightness value. Based on the edge enhancement adaptive threshold, the low grayscale threshold and the high grayscale threshold are calculated; Using the low grayscale threshold and the high grayscale threshold, the de-reflected image is divided into three grayscale histograms; Establish initial probability density functions for three gray-level histograms, and then modify the three initial probability density functions to obtain three modified probability density functions; The cumulative distribution functions of the three modified probability density functions are calculated to obtain the enhancement functions of the three gray-level histograms; Obtain the maximum grayscale value in the de-reflected image; For any pixel in the de-reflective image, edge enhancement is performed on the pixel based on the maximum gray value and the enhancement function corresponding to the gray histogram to which the pixel belongs, to obtain the enhanced pixel. After all pixels in the de-reflective image have been polled, the enhanced pixels are used to form an enhanced image.

[0013] In one possible design, an edge enhancement adaptive threshold is calculated based on the average pixel energy, the maximum brightness value, the minimum brightness value, and the average brightness value, including: The edge enhancement adaptive threshold is calculated using the following formula; ; In the formula, This represents the edge enhancement adaptive threshold. This represents the average brightness value. These represent the maximum and minimum brightness values, respectively. This represents the average pixel energy; Accordingly, based on the edge enhancement adaptive threshold, the low grayscale threshold and high grayscale threshold are calculated, including: The low grayscale threshold and the high grayscale threshold are calculated according to the following formula; ; In the formula, These represent the high grayscale threshold and the low grayscale threshold, respectively. This represents the compensation coefficient.

[0014] In one possible design, the three initial probability density functions are modified to obtain three modified probability density functions, including: For any initial probability density function, obtain the maximum probability value, minimum probability value, and correction coefficient; Calculate the probability difference between the maximum and minimum probability values; Based on the probability difference, the maximum probability value, the minimum probability value, and the correction coefficient, the initial probability density function is modified to obtain the modified probability density function corresponding to the initial probability density function.

[0015] Secondly, a computer vision-based automatic empty bucket counting and warehousing verification system is provided, including: The acquisition unit is used to acquire images of empty buckets within the delivery station. An image segmentation unit is used to perform region segmentation processing on the empty bucket image to segment out the reflective area and the non-reflective area of ​​the bucket body in the empty bucket image. The reflective compensation unit is used to calculate the reflective compensation amount of each reflective pixel in the reflective area based on the reflective area and the non-reflective area of ​​the barrel body. The reflection compensation unit is also used to perform reflection compensation processing on each reflection pixel in the empty bucket image based on the reflection compensation amount of each reflection pixel to obtain a de-reflected image. An edge enhancement unit is used to perform adaptive edge enhancement processing on the de-reflected image to obtain an enhanced captured image; The edge segmentation unit is used to perform edge segmentation on the enhanced captured image to obtain the edge image of each empty bucket in the empty bucket captured image; The label recognition unit is used to identify brand labels on each edge image to obtain brand information for each empty bucket. The counting unit is used to determine the number of empty buckets of different brands in the distribution station based on the edge image and brand information of each empty bucket, so as to verify the empty buckets entering the warehouse according to the number of empty buckets of different brands.

[0016] Thirdly, a computer vision-based automatic empty bucket inventory and warehousing verification device is provided. Taking the device as an electronic device as an example, it includes a memory, a processor, and a transceiver that are connected in sequence. The memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer programs and execute the computer vision-based automatic empty bucket inventory and warehousing verification method as described in the first aspect or any possible design of the first aspect.

[0017] Fourthly, a storage medium is provided, on which instructions are stored, which, when executed on a computer, perform the computer vision-based automatic empty bucket inventory and warehousing verification method as described in the first aspect or any possible design of the first aspect.

[0018] Fifthly, a computer program product containing instructions is provided, which, when executed on a computer, causes the computer to perform the computer vision-based automatic empty bucket inventory and warehousing verification method as described in the first aspect or any possible design of the first aspect.

[0019] Beneficial effects: (1) This invention segments the reflective areas of an image taken of an empty bucket and calculates the reflection compensation amount for each reflective pixel using non-reflective and reflective areas. Then, based on the reflection compensation amount, targeted compensation is applied to the reflective areas, thereby effectively eliminating reflection interference and significantly improving image quality, laying a solid foundation for subsequent accurate edge segmentation. At the same time, to address the edge blurring problem that may occur during image acquisition and transmission, this invention further performs adaptive edge enhancement processing on the basis of de-reflection. In this way, this operation can effectively highlight the edge features of the bucket body, making the subsequent edge segmentation results more accurate. The invention ensures completeness, significantly improving the accuracy of identifying the outline of each empty bucket. Furthermore, by accurately extracting the edge image of each empty bucket and combining it with brand label recognition, the invention can complete the quantity counting and brand classification of empty buckets in one go. This allows for the determination of the quantity of empty buckets from different brands, which can then be directly compared and verified with SaaS deposit invoices. This process completely replaces the inefficient traditional method of manually counting, identifying, and entering data one by one, achieving automation and intelligence in the empty bucket warehousing process and solving the long-standing business bottleneck that has constrained site operational efficiency. Therefore, this invention is highly suitable for large-scale application and promotion. Attached Figure Description

[0020] Figure 1 A flowchart illustrating the steps of the computer vision-based automatic empty bucket inventory and warehousing verification method provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the edge segmentation model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a convolutional block provided in an embodiment of the present invention; Figure 4 A structural diagram of the computer vision-based automatic empty bucket counting and warehousing verification system provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0022] It should be understood that although the terms first, second, etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit, without departing from the scope of the exemplary embodiments of the invention.

[0023] It should be understood that the term "and / or" that may appear in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" that may appear in this document describes another relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this document generally indicates that the related objects before and after it are in an "or" relationship.

[0024] Example: See Figure 1 As shown, the computer vision-based automatic empty bucket counting and warehousing verification method provided in this embodiment can be executed by, but is not limited to, computer devices with certain computing resources, such as servers, edge computers, personal computers (PCs, which are multi-purpose computers of a size, price, and performance suitable for personal use; desktop computers, laptops, mini-laptops, tablets, and ultrabooks are all personal computers), smartphones, or personal digital assistants (PDAs). It is understood that the aforementioned executing entities do not constitute a limitation on the embodiments of this application. Accordingly, the operation steps of this method can be, but are not limited to, the steps S1 to S8 below.

[0025] S1. Obtain images of empty buckets within the delivery station; in specific implementations, this can be achieved, but is not limited to, using a smartphone to photograph empty buckets within the delivery station, thereby obtaining images of the empty buckets (which are RGB images); after obtaining the images of the empty buckets, de-reflection processing is required to effectively eliminate the negative impact of reflections on the smooth surface of the bucket on contour recognition; in this embodiment, the reflective area and the non-reflective area of ​​the bucket are first segmented, and then, based on this, the reflection compensation amount of each reflective pixel in the reflective area is calculated. Finally, based on the aforementioned reflection compensation amount, the reflection is eliminated. The process can be, but is not limited to, the steps S2 to S4 described below.

[0026] S2. Perform region segmentation processing on the empty bucket image to separate the reflective area and the non-reflective area of ​​the bucket body in the image. In specific applications, this embodiment uses clustering to identify the pixels in the reflective and non-reflective areas of the empty bucket image, obtaining different pixel clusters. Then, based on the clustered pixel clusters, the reflective area and the non-reflective area of ​​the bucket body are segmented. The process can be, but is not limited to, the steps S21 to S25 below.

[0027] S21. Calculate the chromaticity mapping vector for each pixel in the empty bucket image, and use the chromaticity mapping vectors of each pixel to map the RGB values ​​of each pixel in the empty bucket image to a chromaticity unit sphere to obtain a mapped image. In specific implementation, for any pixel, for example, but not limited to, first calculate the squares of the R channel value, the G channel value, and the B channel value of that pixel, then sum the squares of the R channel value, the G channel value, and the B channel value to obtain a sum of squares; then, take the square root of the sum of squares and take the positive result to obtain the mapping parameter; finally, divide the RGB value of that pixel by the mapping parameter to obtain the chromaticity mapping vector of that pixel (which also contains the chromaticity mapping values ​​on the R, G, and B channels). In this way, after obtaining the chromaticity mapping vector, the RGB value of that pixel can be mapped to a chromaticity unit sphere. Based on this, process the remaining pixels in the empty bucket image in the aforementioned manner to obtain a mapped image.

[0028] Based on the aforementioned operation method, pixels can be mapped onto a unit sphere to eliminate the influence of light intensity and retain pure color information. Thus, after obtaining the mapped image, color difference enhancement can be performed, as shown in step S22 below.

[0029] S22. Perform an exponential transformation on the mapped image to obtain the transformed image. In specific applications, the chromaticity of the reflective area and the material area (i.e., the non-reflective area of ​​the barrel) in the image of an empty barrel may be similar. The exponential transformation can stretch the chromaticity space, enhance the chromaticity difference between different material areas, expand the inter-class distance, and make the clustering effect more significant, thereby improving the clustering accuracy. For example, but not limited to, the following formula can be used to perform the exponential transformation.

[0030] ; In the formula, This represents the exponential transform graph, where j represents the adjustment exponent. The transformed image is obtained by substituting the values ​​of the chromaticity mapping vectors of each pixel in the transformed image into the aforementioned formula, thus forming the transformed pixel information.

[0031] After obtaining the transformed image, clustering processing can be performed to cluster the pixels in the entire transformed image into two clusters, as shown in step S23 below.

[0032] S23. Perform clustering processing on each pixel in the transformed image to obtain two pixel clusters; in specific implementation, for example, the number of clusters is 2, and the k-means clustering algorithm can be used, but is not limited to, to perform pixel clustering to obtain two pixel clusters; of course, k-means clustering is a commonly used clustering technique, and its principle will not be elaborated here.

[0033] After obtaining two pixel clusters, cluster type identification can be performed to distinguish which cluster contains reflective pixels and which contains non-reflective pixels. The cluster identification process is shown in step S24 below.

[0034] S24. Obtain the light source chromaticity vector and calculate the Euclidean distance between the cluster center of two pixel clusters and the light source chromaticity vector. In specific implementation, under natural light or common indoor light sources, the light source chromaticity is usually close to pure white or pale yellow. Therefore, the light source setting vector can be preset to (0.33, 0.33, 0.33). The chromaticity of the reflective area will be very close to the light source chromaticity. In this way, cluster classification can be performed based on the distance between the cluster center of two pixels and the light source chromaticity vector. The process is shown in step S25 below.

[0035] S25. Based on the Euclidean distance between the cluster centers of the two pixel clusters and the chromaticity vector of the light source, the two pixel clusters are divided into a reflective cluster and a non-reflective cluster. Based on the reflective and non-reflective clusters, the reflective area and the non-reflective area of ​​the bucket body are segmented from the image of the empty bucket. In this embodiment, after calculating the Euclidean distance between the cluster centers of the two pixel clusters and the chromaticity vector of the light source, the pixel cluster with the smallest distance can be considered the reflective cluster, meaning the pixels in this cluster can be considered reflective pixels. The remaining pixel cluster is considered the non-reflective cluster. Based on this, after completing the cluster classification, the region can be segmented based on the pixels in the reflective and non-reflective clusters to obtain the reflective area and the non-reflective area of ​​the bucket body.

[0036] After segmenting the reflective area and the non-reflective area of ​​the bucket body in the empty bucket image through the aforementioned steps S21 to S25, the reflection compensation amount of each reflective pixel in the reflective area can be calculated based on this, as shown in step S3 below.

[0037] S3. Based on the reflective area and the non-reflective area of ​​the barrel, calculate the reflection compensation amount of each reflective pixel in the reflective area. In specific implementation, to eliminate the reflection on the surface of the empty barrel, it is necessary to first obtain the inherent color of the empty barrel material itself, and then restore the pixel value of the reflective area based on the inherent color of the empty barrel material. Therefore, in this embodiment, the inherent color of the empty barrel material is obtained by using the non-reflective area of ​​the barrel body, and thus it is used as the second reflection suppression amount. At the same time, the non-reflective amount of the pixels in the reflective area (i.e., the first reflection suppression amount) is calculated by using the reflective area. Finally, the reflection compensation amount of each reflective pixel can be obtained by combining the two. The aforementioned calculation process can be, but is not limited to, as shown in steps S31 to S36 below.

[0038] S31. For any reflective pixel in the reflective area, map the RGB value of the reflective pixel onto the chromaticity unit sphere to obtain the chromaticity mapping vector of the reflective pixel; in specific implementation, the calculation process of the chromaticity mapping vector can be referred to the aforementioned step S21, and its principle will not be repeated.

[0039] After calculating the chromaticity mapping vector of any reflective pixel, the chromaticity compensation parameters can be calculated based on this, as shown in step S32 below.

[0040] S32. Based on the chromaticity mapping vector, calculate the chromaticity compensation parameter of any reflective pixel. In specific applications, this embodiment calculates the chromaticity compensation parameter of any reflective pixel based on the principle of orthogonal decomposition of light source chromaticity. In layman's terms, it is to perform vector decomposition on the chromaticity mapping vector of any reflective pixel based on the chromaticity direction of the light source. Optionally, the specific calculation process of the chromaticity compensation parameter may be, but is not limited to, the steps S32a to S32i below.

[0041] S32a. Obtain the light source chromaticity vector, and calculate the projection coefficient of the pixel color of the any reflective pixel in the direction of the light source based on the light source chromaticity vector and the chromaticity mapping vector of the any reflective pixel. In specific implementation, for example, but not limited to, the light source chromaticity vector and the chromaticity mapping vector can be multiplied by a dot product to obtain the projection coefficient of the pixel color of the any reflective pixel in the direction of the light source. Then, the projection coefficient can be used to project the chromaticity mapping vector of the any reflective pixel, as shown in step S32b below.

[0042] S32b. Using the projection coefficient, the light source chromaticity vector, and the chromaticity mapping vector, calculate the projection vector of any reflective pixel on a plane orthogonal to the light source direction. In this embodiment, for example, but not limited to, multiplying the projection coefficient by the aforementioned light source chromaticity vector to obtain an intermediate vector; then, subtracting the intermediate vector from the chromaticity mapping vector to obtain the projection vector of any reflective pixel on a plane orthogonal to the light source direction. This vector represents the inherent color of the object itself after excluding the influence of the light source.

[0043] Thus, after calculating the projection vector, the initial chromaticity compensation parameters can be calculated by combining the chromaticity mapping vector and the projection coefficients, as shown in steps S32c to S32g below.

[0044] S32c. From the chroma mapping vector, select the maximum and minimum mapping values; in this embodiment, it has been explained above that the chroma mapping vector contains chroma mapping values ​​on the R channel, G channel and B channel. Therefore, it is equivalent to selecting the maximum and minimum mapping values ​​from the chroma mapping values ​​of the three channels; then, based on this, the first chroma intermediate parameter can be calculated, the process of which is shown in step S32d below.

[0045] S32d. Calculate the difference between the maximum and minimum mapping values, and use the ratio between the maximum mapping value and the difference as the first intermediate chromaticity parameter.

[0046] After calculating the first intermediate chromaticity parameter, the magnitude of the projection vector can be obtained. Then, based on the magnitude, the second intermediate chromaticity parameter is calculated, as shown in steps S32e and S32f below.

[0047] S32e. Calculate the magnitude of the projection vector and obtain the product between the magnitude and the difference; after calculating the difference between the magnitude and the aforementioned maximum and minimum mapping values, the two can be multiplied; then, the second chromaticity intermediate parameter can be calculated based on the product of the two, as shown in step S32f below.

[0048] S32f. The ratio between the projection coefficient and the product is used as the second chromaticity intermediate parameter.

[0049] After calculating the second intermediate chromaticity parameter, it can be added to the first intermediate chromaticity parameter to obtain the initial chromaticity compensation parameter. The process is shown in step S32g below.

[0050] S32g. Summate the first chromaticity intermediate parameter and the second chromaticity intermediate parameter to obtain the initial chromaticity compensation parameter.

[0051] Based on the aforementioned steps, after calculating the initial chromaticity compensation parameters, the final chromaticity compensation parameters can be calculated by combining the difference between the maximum and minimum mapping values, as well as the maximum mapping value. The process is shown in steps S32h to S32i below.

[0052] S32h. Calculate the product between the initial chromaticity compensation parameter and the difference value to obtain the third intermediate chromaticity parameter.

[0053] S32i. The difference between the maximum mapping value and the third chromaticity intermediate parameter is used as the chromaticity compensation parameter.

[0054] Therefore, after calculating the color compensation parameters through the aforementioned steps S32a to S32i, the first reflection suppression amount, which is the amount of no reflection of any reflective pixel, can be calculated based on these parameters. The process is shown in step S33 below.

[0055] S33. Using the chromaticity compensation parameter and the chromaticity mapping vector, calculate the first reflection suppression amount of any reflective pixel, and after traversing all reflective pixels in the reflective area, obtain the first reflection suppression amount of each reflective pixel; in specific implementation, for example, but not limited to, first calculate the difference between the chromaticity mapping values ​​of the R channel, G channel and B channel and the chromaticity compensation parameter to obtain the channel suppression amount of the R channel, G channel and B channel respectively; then, use the channel suppression amounts of the R channel, G channel and B channel to form the first reflection suppression amount.

[0056] Thus, after calculating the first reflection suppression amount of any reflective pixel, the inherent color of the empty barrel material can be calculated based on the non-reflective area of ​​the barrel body, as shown in steps S34 and S35 below.

[0057] S34. Based on the non-reflective area of ​​the barrel body, calculate the average brightness of the barrel body and the chromaticity direction component of the barrel body. In specific implementation, as explained above, this is achieved by clustering pixels to obtain two clusters: one for reflective points and one for non-reflective points. Therefore, the pixels in the non-reflective point cluster are the pixels in the non-reflective area of ​​the barrel body. Based on this, the magnitude of the RGB vector of each pixel in the non-reflective point cluster can be calculated first to obtain the brightness of each pixel. Then, the average brightness of all pixels is taken as the average brightness of the barrel body. At the same time, the average RGB vector of each pixel in the non-reflective point cluster is calculated to obtain the mean vector. Next, the L2 norm of the mean vector is calculated to obtain the normalization constant. Finally, the mean vector is divided by the normalization constant to obtain the chromaticity direction component. In this way, the chromaticity direction component is used to provide the correct color ratio.

[0058] After calculating the average brightness of the barrel and the chromaticity direction component of the barrel, the inherent chromaticity of the empty barrel material can be calculated based on this, as shown in step S35 below.

[0059] S35. Calculate the second reflection suppression amount using the average brightness of the barrel body and the chromaticity direction component; in this embodiment, the second reflection suppression amount can be obtained by multiplying the average brightness of the barrel body and the chromaticity direction component; then, the reflection compensation amount of each reflective pixel can be formed by combining the first reflection suppression amount of each reflective pixel, as shown in step S36 below.

[0060] S36. Using the second reflection suppression amount and the first reflection suppression amount of each reflective pixel, a reflection compensation amount for each reflective pixel in the reflective region is formed.

[0061] Therefore, after calculating the reflection compensation amount of each reflective pixel in the reflective area through the aforementioned steps S31 to S36, reflection compensation can be performed, as shown in step S4 below.

[0062] S4. Based on the reflection compensation amount of each reflective pixel, reflection compensation processing is performed on each reflective pixel in the empty bucket image to obtain a de-reflective image. In specific implementation, for any reflective pixel, for example, but not limited to, a weighted summation of the second reflection suppression amount and the first reflection suppression amount of the any reflective pixel (both with a weight of 0.5) can be performed to obtain the reflection-compensated pixel value of the any reflective pixel. Thus, in the aforementioned manner, after all reflective pixels in the empty bucket image have been polled and compensated, the pixel value of each reflective pixel can be updated to the reflection-compensated pixel value, and at this time, the de-reflective image can be obtained.

[0063] Thus, through the aforementioned steps S2 to S4, the present invention segments the reflective area of ​​the empty barrel image and uses the pixel information of the non-reflective area to specifically compensate the reflective area. In this way, reflective interference can be effectively eliminated, thereby significantly improving the image quality and laying a solid foundation for subsequent accurate edge segmentation.

[0064] Meanwhile, during image acquisition and transmission, there may be parts of the image with indistinct edges, which can easily interfere with edge contour extraction. Therefore, after completing reflection compensation, this embodiment also includes an adaptive edge enhancement step, the process of which is shown in step S5 below.

[0065] S5. Perform adaptive edge enhancement processing on the de-reflected image to obtain an enhanced captured image; in specific applications, this embodiment provides an edge enhancement algorithm in three dimensions: partition adaptation, probability correction, and curve optimization, to address issues such as unclear edge details and blurriness. The process can be, but is not limited to, the steps S51 to S59 below.

[0066] S51. Based on the de-reflected image, calculate the average pixel energy of the de-reflected image; in specific implementations, for example, but not limited to, wavelet decomposition can be performed on the de-reflected image, and then the high-frequency energy of each pixel in the horizontal, vertical and diagonal directions can be obtained; then, the three high-frequency energies are squared and summed, and then the square root is taken to obtain the energy of each pixel; finally, the mean of the energy of all pixels is taken as the average pixel energy.

[0067] After obtaining the average pixel energy, the maximum brightness value, minimum brightness value and average brightness value in the reflected image can be obtained, as shown in step S52 below.

[0068] S52. Obtain the maximum brightness value, minimum brightness value, and average brightness value of the de-reflected image; in this embodiment, the ITU-R BT.601 standard weighted formula can be used to calculate the brightness of each pixel in the reflective image; thus, after obtaining the brightness of each pixel, the maximum brightness value, minimum brightness value, and average brightness can be determined; then, based on this, the edge enhancement adaptive threshold can be calculated, as shown in step S53 below.

[0069] S53. Calculate the edge enhancement adaptive threshold based on the average pixel energy, the maximum brightness value, the minimum brightness value, and the average brightness value. In specific applications, traditional histogram equalization performs global statistics on the entire image. However, if the image contains both extremely dark and extremely bright areas (such as backlit photos), global equalization often leads to overexposure in dark areas (noise amplification) or overexposure in bright areas (loss of detail). Therefore, this embodiment introduces an edge enhancement adaptive threshold to perform dynamic partitioning, thereby avoiding the aforementioned drawbacks of global equalization.

[0070] For example, but not limited to, the following formula can be used to calculate the aforementioned edge enhancement adaptive threshold.

[0071] ; In the formula, This represents the edge enhancement adaptive threshold. This represents the average brightness value. These represent the maximum and minimum brightness values, respectively. This represents the average pixel energy.

[0072] Thus, based on the aforementioned formula, after calculating the edge enhancement adaptive threshold, the low grayscale threshold and the high grayscale threshold can be determined, as shown in step S54 below.

[0073] S54. Based on the edge enhancement adaptive threshold, calculate the low grayscale threshold and the high grayscale threshold; in specific implementation, for example, but not limited to, the following formula can be used to calculate the aforementioned low grayscale threshold and high grayscale threshold.

[0074] ; In the formula, These represent the high grayscale threshold and the low grayscale threshold, respectively. This represents the compensation coefficient.

[0075] Thus, as shown in the aforementioned formula, this embodiment automatically determines the brightness tone of the image by introducing an edge enhancement adaptive threshold and calculates two dynamic thresholds to perform grayscale segmentation of the de-reflective image. Therefore, compared to traditional histogram equalization, this invention strictly partitions the image, equalizing dark areas within dark areas and bright areas within bright areas. This avoids "grayscale contention" between different brightness regions, prevents the grayscale levels of dark areas from being "occupied" by bright areas, thereby protecting the original brightness levels of each region and allowing edge information to be naturally enhanced within its original region.

[0076] Meanwhile, in blurred images, edges are often located in areas where grayscale changes are slow, and their pixel grayscale values ​​are concentrated in a very narrow range. In the global histogram, the probability of these detailed ranges is extremely low, and they are easily submerged by the statistical information of the background, resulting in an insignificant enhancement effect. The edge enhancement adaptive threshold of this invention essentially reflects the overall brightness tone of the image. Therefore, when the calculation result shows that the image is too dark or too bright, the algorithm will adjust the high-illuminance and low-illuminance areas in a targeted manner, thereby ensuring that the low-illuminance areas (usually where the edges are hidden) are separated, thus improving the visibility of weak edges.

[0077] Furthermore, the edge enhancement adaptive threshold is calculated based on the actual brightness of the image. For images captured by cameras with different dynamic ranges, the algorithm can automatically sense and adjust the partition threshold, enhancing the universality of the method. Moreover, the introduction of the edge enhancement adaptive threshold makes the partition boundary not fixed, but smoothly moving according to the image content. Therefore, in conjunction with subsequent probability density correction and curve correction, the grayscale transition at the partition boundary can be relatively smooth, thereby enhancing the target edge while suppressing the false edges introduced by the algorithm itself to the greatest extent.

[0078] After calculating the low grayscale threshold and the high grayscale threshold, the de-reflected image can be divided, as shown in step S55 below.

[0079] S55. Using the low grayscale threshold and the high grayscale threshold, the de-reflected image is divided into three grayscale histograms. In a specific implementation, the de-reflected image is grayscale processed to obtain a grayscale image. Then, pixels with grayscale values ​​less than or equal to the low grayscale threshold are assigned to the low illumination region, pixels with grayscale values ​​greater than or equal to the high grayscale threshold are assigned to the high illumination region, and the remaining pixels are assigned to the medium illumination region. In this way, by statistically analyzing the grayscale values ​​of the pixels in the three illumination regions, three grayscale histograms can be obtained.

[0080] After obtaining the three gray-level histograms, the initial probability density function and the correction of the density function can be performed, as shown in step S56 below.

[0081] S56. Establish the initial probability density functions for three gray-level histograms, and then modify the three initial probability density functions to obtain three modified probability density functions.

[0082] In practical implementation, the initial probability density function of any gray-level histogram can be expressed as: In the formula, This represents the initial probability of gray level z appearing. This represents the number of times the gray level z appears in any gray level histogram (i.e., the number of pixels with a gray level value of z in any gray level histogram). This represents the total number of pixels corresponding to any given grayscale histogram.

[0083] Thus, after calculating the initial probability density function of each gray-level histogram, function correction can be performed. The reason is that in traditional histogram equalization, the mapping function is entirely determined by the original probability density, meaning that a gray level that appears more frequently will occupy a wider gray-level range after enhancement. However, this logic has the following problem in blurred edge images: edges (blurred contours) typically occupy only a small number of pixels. If the initial probability density function is used directly, a large number of gray levels will be allocated to the background after equalization, while the gray levels where the edges are located will be compressed, resulting in unclear edges. Therefore, this embodiment performs function correction, the process of which is as follows: First, for any initial probability density function, obtain the maximum probability value, the minimum probability value, and the correction coefficient; then, calculate the probability difference between the maximum and minimum probability values; finally, based on the probability difference, the maximum probability value, the minimum probability value, and the correction coefficient, the initial probability density function can be corrected to obtain the corrected probability density function corresponding to the initial probability density function.

[0084] For example, the initial probability density function can be modified according to the following formula, but is not limited to.

[0085] ; In the formula, This represents the modified probability density function corresponding to any given initial probability density function. These represent the maximum and minimum probability values ​​(set values), respectively. The probability difference is... Denotes any given initial probability density function. This represents the correction factor.

[0086] As can be seen from the above formula, by introducing the maximum and minimum probability values, a modified probability density function based on an inverse proportional function is constructed. That is, the numerator is a constant and the denominator is the initial probability density function. Therefore, a high original probability (background, flat areas) results in a low modified probability (suppressed), and vice versa. Based on this, the core logic of the formula is: the higher the original probability of a gray level, the lower the modified weight; the lower the original probability of a gray level (usually edges, textures, and other details), the higher the modified weight. In this way, the purpose of suppressing high-frequency gray levels and improving low-frequency gray levels can be achieved.

[0087] After correcting the initial probability density function, the cumulative distribution function can be calculated, as shown in step S57 below.

[0088] S57. Calculate the cumulative distribution function of the three modified probability density functions to obtain the enhancement function of the three gray-level histograms; in this embodiment, the cumulative distribution function corresponding to any modified probability density function is: ; In the formula, This represents the cumulative distribution value of the gray level z. Let c be the corrected probability density value of gray level c. Thus, the cumulative distribution function is equivalent to the sum of probabilities accumulated from gray level 1 to z.

[0089] Thus, based on the aforementioned formula, after obtaining the three cumulative distribution functions, the conversion from probability distribution to grayscale mapping is realized; then, edge enhancement can be performed based on this, as shown in steps S58 and S59 below.

[0090] S58. Obtain the maximum grayscale value in the de-reflected image.

[0091] S59. For any pixel in the de-reflective image, based on the maximum gray value and the enhancement function corresponding to the gray histogram to which the pixel belongs, edge enhancement is performed on the pixel to obtain the enhanced pixel. After all pixels in the de-reflective image have been polled, the enhanced pixels are used to form an enhanced image.

[0092] In specific implementation, for example, but not limited to, the following formula, can be used to enhance the edge of any pixel.

[0093] ; In the formula, Represents any pixel after enhancement. This represents the enhancement function corresponding to the gray-level histogram to which any pixel belongs (i.e., the cumulative probability sum of the gray levels corresponding to any pixel). This represents the correction factor. Indicates the maximum grayscale value. This indicates the correction enhancement factor.

[0094] Therefore, in this embodiment, after obtaining the cumulative distribution function, it does not directly use the function for grayscale mapping. Instead, it introduces gamma correction for curve optimization, which brings the following advantages: (1) By correcting the enhancement factor, the enhanced image can be made more in line with the visual habits of the human eye. For example, when the correction enhancement factor is less than 1, the contrast of the dark area is stretched, making the blurry edges hidden in the shadow easier for the human eye to perceive; when it is greater than 1, the details of the bright area are enhanced. This visual adaptability makes the processed image not only better in mathematical indicators, but also clearer in subjective visual perception.

[0095] (2) The cumulative distribution function has completed the redistribution of grayscale resources (background compression, edge stretching), but its mapping curve is fixed. Gamma correction provides an additional degree of freedom, allowing nonlinear adjustment of the shape of the mapping curve. This fine-tuning capability enables the algorithm to adapt to a wider range of image types and application scenarios, enhancing the robustness of the method.

[0096] (3) Traditional histogram equalization sometimes produces an "over-enhancement" effect, making the image look stiff, unnatural, or even with false edges. The introduction of gamma correction plays a smoothing and buffering role. Therefore, by setting the correction enhancement factor reasonably, gray-level abrupt changes can be avoided while maintaining the enhancement effect.

[0097] (4) The correction enhancement factor is based on the sub-histogram partitioning, which means that different enhancement factors can be used for each brightness region (low illuminance, intermediate illuminance, high illuminance). Thus, this partitioned adaptive correction makes the enhancement effect more refined and truly achieves the goal of adapting to local conditions.

[0098] (5) The generated enhanced image has clearer edge gradients and a cleaner background after the mapping curve is corrected and optimized, which can improve the accuracy of subsequent edge extraction.

[0099] After edge enhancement is completed through the aforementioned steps S51 to S59, edge segmentation of the empty buckets in the image can be performed, as shown in step S6 below.

[0100] S6. Perform edge segmentation on the enhanced image to obtain the edge image of each empty bucket in the empty bucket image; in specific implementations, for example, but not limited to, the enhanced image can be input into the edge segmentation model to output the edge image of each empty bucket.

[0101] Furthermore, this embodiment provides an edge segmentation model, see [link to relevant documentation]. Figure 2 As shown, it may include, but is not limited to, an encoder, an attention mechanism layer, a feature fusion layer, and a decoder. The encoder includes multiple layers of convolutional blocks, and the decoder includes multiple layers of deconvolutional blocks. The encoder and decoder have the same number of layers (4 layers of convolutional blocks and 4 layers of deconvolutional blocks can be set), and the i-th layer of convolutional blocks in the encoder and the i-th layer of deconvolutional blocks in the decoder are connected in a skip connection. At the same time, the decoder performs data transmission from bottom to top, that is, the bottommost deconvolutional block receives the output of the feature fusion layer, and then passes it to the next layer of deconvolutional blocks. Therefore, the topmost deconvolutional block (i.e., the first layer of convolutional blocks) serves as the output of the decoder.

[0102] In practical applications, any convolutional block in the encoder is used to receive the first feature map output by the previous convolutional block, and to extract features from the first feature map to obtain a second feature map which is then input to the next convolutional block and the attention mechanism layer. When any convolutional block is a first-level convolutional block, it is used to receive the enhanced captured image.

[0103] Thus, the encoder consists of four stacked convolutional blocks that progressively output multi-level feature maps ranging from low-level texture features (such as edges and corners) to high-level semantic features (such as object parts and overall outlines).

[0104] Optional, see Figure 3 As shown, any convolutional block can include, but is not limited to, a first convolutional layer, a batch normalization layer, and a second convolutional layer connected in sequence, with the input of the first convolutional layer and the output of the second convolutional layer connected via a residual connection. Thus, when any convolutional block receives the first feature map, it first performs convolution processing using the first convolutional layer (kernel size 3×3, stride 1). Then, the convolution result enters the batch normalization layer for batch normalization. Next, the ReLU activation function is used to activate the convolution, introducing nonlinearity, enhancing sparsity, accelerating the training process, and mitigating the gradient vanishing problem. Then, the second convolutional layer performs a second convolution, also with batch normalization. Finally, the input of any convolutional block is element-wise added to the second batch normalization result, thus achieving a residual connection. Similarly, the added result is passed through the ReLU activation function to obtain the final output of any convolutional block.

[0105] In this way, residual connections allow gradients to be directly backpropagated from deep layers to shallow layers, effectively alleviating the gradient vanishing problem and enabling the network to be trained deeper, thereby extracting more abstract and semantically stronger features.

[0106] After the enhanced image is passed through four convolutional blocks in the coding layer to output four different levels of second feature maps, an attention mechanism layer can be used to focus on important edge regions, i.e.: The attention mechanism layer is used to perform max pooling and average pooling on the second feature map output by each convolutional block, and concatenate the max pooling results and average pooling results to obtain concatenated features. The concatenated features are then processed by a convolutional layer to obtain convolutional features, which are then activated using the Sigmoid activation function to obtain the channel attention weights of each second feature map.

[0107] Simultaneously, the attention mechanism layer is used to perform global average pooling and global max pooling on the second feature map output by each convolutional block, and inputs the results of global average pooling and global max pooling into two multilayer perceptrons sharing network parameters to obtain two feature vectors respectively. Then, the two feature vectors are added together and activated by the Sigmoid activation function to obtain the spatial attention weights of each second feature map. Finally, the channel attention weights and spatial attention weights of each second feature map can be used to enhance the features of each second feature map to obtain enhanced second feature maps (the channel attention weights, spatial attention weights, and second feature maps are multiplied to obtain the enhanced second feature maps). In this way, important edge pixels and related feature channels are highlighted, and background noise is suppressed.

[0108] After feature enhancement is completed, multi-scale fusion can be performed, namely: a feature fusion layer, which is used to perform global average pooling on each enhanced second feature map to obtain multiple global average pooling features. Each global average pooling feature is then input into a multilayer perceptron to obtain the weight features of each enhanced second feature map. In order to normalize each weight feature through the Softmax function, the fusion weights of each enhanced second feature map are obtained.

[0109] In this embodiment, the multilayer perceptron is typically configured as a two-layer fully connected network. Therefore, global average pooling compresses the feature map into a vector of length H. This vector represents the global information descriptor of the feature map at that scale (i.e., the average response of all pixels in each channel). After this vector is fed into the multilayer perceptron, it can be mapped to a single numerical value, which represents the degree of contribution of the feature at this scale to the final edge detection based on the global information of the current feature map. Thus, by feeding the contribution values ​​obtained at different scales into the Softmax function for normalization, the actual contribution of each scale feature to the final edge detection (i.e., mapped to between 0 and 1) can be obtained.

[0110] Thus, this feature fusion layer can address the problem of varying object edge scales in an image by calculating adaptive weights to fuse features from different layers of the encoder. This results in the generation of a comprehensive feature map that integrates multi-level and multi-scale information, making the final fused feature map contain both detailed textures and semantic contours.

[0111] The feature fusion process is as follows: a feature fusion layer is used to perform weighted fusion processing on each enhanced second feature map based on the fusion weight of each enhanced second feature map, so as to obtain a fused feature map output to the decoder.

[0112] After obtaining the fused features, decoding can be performed. The process is as follows: any deconvolution block in the decoder is used to receive the first decoded feature map output by the previous deconvolution block, and based on the first decoded feature map and the second feature map output by the convolution block that is skipped and connected to the deconvolution block, a second decoded feature map is generated and transmitted to the next deconvolution block. When any convolution block is the bottommost (i.e., the 4th layer) deconvolution block, the deconvolution block is used to receive the fused feature map. When any deconvolution block is the topmost (i.e., the first layer) deconvolution block, the output second feature map is convolved by 1 by 1 to obtain the edge image of each empty bucket.

[0113] In this embodiment, the decoder is symmetrical to the encoder, and gradually upsamples the low-resolution feature map back to the original input size through deconvolution (transposed convolution) layers. The structure of any deconvolution block is the same as that of the deconvolution block, except that the convolution layer is replaced with a transposed convolution (i.e., the convolution block is downsampled, while the deconvolution block is upsampled). After the first upsampling in the deconvolution block (which also undergoes batch normalization and activation function), the second feature map output by the convolution block that is skipped to the deconvolution block can be concatenated to achieve feature fusion. Then, a second upsampling is performed, and the second decoded feature map is finally output. Of course, when it is the last deconvolution block, after a 1x1 convolution, the edge image of each empty bucket in the empty bucket image is obtained.

[0114] Therefore, the edge segmentation model provided in this embodiment achieves accurate capture of edge information at different scales by introducing a multi-scale feature fusion mechanism and an adaptive weight adjustment strategy. At the same time, by combining residual connection and attention mechanism enhancement, it can effectively retain the detailed information required for edge detection, thereby solving the problems of edge discontinuity and loss of details in traditional edge detection methods in complex scenes.

[0115] Furthermore, this embodiment provides a loss function that simultaneously considers pixel accuracy and edge continuity; that is, the loss function of the edge segmentation model is: ; In the formula, This represents the total loss of the edge segmentation model. This represents the weighted binary cross-entropy loss (which is a commonly used loss function). These represent the predicted edge image corresponding to the sample image input during model training, and the ground truth edge image corresponding to the sample image, respectively. Represents the gradient operator, Describing the L2 norm, This represents the feature vectors of the predicted and ground truth edge images output by the j-th layer of the VGG network after the predicted and ground truth edge images corresponding to the sample image are input into the pre-trained VGG network. Both represent the loss weight.

[0116] In this embodiment, This represents pixel-level loss, which involves comparing pixels one by one. It is used to determine whether each point is an edge. This represents the edge gradient loss formula, used to calculate the sum of squared differences between the gradients of the predicted edge map and the gradients of the true edge map. Therefore, the role of this loss function is to ensure that the network not only finds edge pixels, but also makes the found edges visually clear and sharp, rather than a blurry mess; and This represents edge-aware loss, which acts as a semantic supervisor to ensure that the edge maps generated by the network are consistent with the real edge maps in terms of high-level structure and object integrity.

[0117] The edge-aware loss is calculated using features proposed by a pre-trained VGG network. The mid-layer features of the VGG network correspond to the parts of an object; therefore, the edge-aware loss forces the predicted edge map to accurately reflect these part features. Figure 1 This ensures that the generated edges are continuous, complete object outlines, rather than scattered fragments.

[0118] Therefore, using the aforementioned loss function that simultaneously considers pixel accuracy and edge continuity for network training can ensure the accuracy of edge extraction by the edge segmentation model.

[0119] Thus, by using the aforementioned edge segmentation model to extract the edge image of each empty bucket, brand label recognition and empty bucket counting can be performed based on the edge image, as shown in steps S7 and S8 below.

[0120] S7. Perform brand label recognition on each edge image to obtain the brand information of each empty bucket. In specific implementation, after obtaining the edge image of each empty bucket, morphological filtering can be performed on the edge image to connect broken contours and remove isolated small noise points to ensure the continuous integrity of the bucket edge. Then, label localization is performed to extract the label region (e.g., by calculating the edge pixel density in the image through a sliding window, and then taking the region corresponding to the sliding window with the highest edge pixel density as the label region, i.e., the label usually contains dense text or pattern edges with high density). Finally, the label region is input into the trained CRNN neural network to perform brand label recognition, thereby obtaining the brand information of each empty bucket. After obtaining the brand information of the empty buckets corresponding to each edge image, empty bucket counting can be performed, as shown in step S8 below.

[0121] S8. Based on the edge image and brand information of each empty bucket, determine the number of empty buckets of different brands in the delivery station, so as to verify the empty bucket entry based on the number of empty buckets of different brands. In this embodiment, after morphological filtering of the edge image, contour extraction can be performed, that is, each contour corresponds to a connected region. Then, filter conditions are set according to the geometric features of the empty bucket in the image to exclude interference contours that are not the bucket body. The geometric features include: area, perimeter, roundness, aspect ratio, and convexity (the ratio of the convex hull area of ​​the contour to the actual area). Finally, the connected regions are filtered through the aforementioned geometric features, that is, standard values ​​are set for area, perimeter, roundness, aspect ratio, and convexity, and the connected regions that meet the aforementioned geometric features are taken as the true empty bucket contours. In this way, by counting the true empty bucket contours, the number of empty buckets can be obtained. At the same time, combined with the brand information corresponding to each edge image identified above, the number of empty buckets of different brands can be obtained. After obtaining the number of empty buckets of different brands, it can be compared with the SaaS deposit bill to realize the empty bucket entry verification.

[0122] Therefore, through the computer vision-based automatic empty bucket counting and warehousing verification method described in detail in steps S1 to S8 above, this invention specifically addresses the image reflection interference problem in the complex environment of delivery stations. It performs reflective area segmentation and compensation processing on the empty bucket images, effectively eliminating the negative impact of reflection on the smooth surface of the bucket on contour recognition. Furthermore, it adaptively enhances the de-reflected image, significantly improving the extraction accuracy of edge features for each empty bucket. Then, while accurately identifying the edge images of each empty bucket, it completes brand label classification, thereby determining the quantity of empty buckets from different brands at once for direct comparison and verification with the SaaS deposit invoice. Thus, this invention completely replaces the inefficient traditional manual counting, identification, and data entry method, greatly improving the efficiency and accuracy of empty bucket warehousing, avoiding errors or omissions due to fatigue or negligence, effectively ensuring the reliability of deposit settlement, and realizing the automation and intelligence of the empty bucket warehousing process, solving the long-standing business bottleneck problem that restricts the operational efficiency of stations.

[0123] like Figure 4 As shown, the second aspect of this embodiment provides a hardware system for implementing the computer vision-based automatic empty bucket inventory and warehousing verification method described in the first aspect of the embodiment, comprising: The acquisition unit is used to acquire images of empty buckets within the delivery station.

[0124] The image segmentation unit is used to perform region segmentation processing on the image of the empty bucket to separate the reflective area and the non-reflective area of ​​the bucket body in the image of the empty bucket.

[0125] The reflective compensation unit is used to calculate the reflective compensation amount of each reflective pixel in the reflective area based on the reflective area and the non-reflective area of ​​the barrel.

[0126] The reflection compensation unit is also used to perform reflection compensation processing on each reflective pixel in the empty bucket image based on the reflection compensation amount of each reflective pixel to obtain a de-reflective image.

[0127] An edge enhancement unit is used to perform adaptive edge enhancement processing on the de-reflected image to obtain an enhanced captured image.

[0128] The edge segmentation unit is used to perform edge segmentation on the enhanced captured image to obtain the edge image of each empty bucket in the empty bucket captured image.

[0129] The label recognition unit is used to identify brand labels on each edge image to obtain brand information for each empty bucket.

[0130] The counting unit is used to determine the number of empty buckets of different brands in the distribution station based on the edge image and brand information of each empty bucket, so as to verify the empty buckets entering the warehouse according to the number of empty buckets of different brands.

[0131] The working process, working details and technical effects of the system provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0132] like Figure 5 As shown, the third aspect of this embodiment provides an automatic empty bucket counting and warehousing verification device based on computer vision. Taking the device as an electronic device as an example, it includes: a memory, a processor, and a transceiver that are connected in sequence. The memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the automatic empty bucket counting and warehousing verification method based on computer vision as described in the first aspect of the embodiment.

[0133] For specific examples, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or first-in-last-out (FILO) memory, etc.; specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.

[0134] In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. For example, the processor may not be limited to microprocessors of the STM32F105 series, reduced instruction set computer (RISC) microprocessors, x86 architecture processors, or processors with integrated neural network processing units (NPUs). The transceiver may be, but is not limited to, a Wi-Fi transceiver, a Bluetooth transceiver, a General Packet Radio Service (GPRS) transceiver, a ZigBee (a low-power LAN protocol based on the IEEE 802.15.4 standard) transceiver, a 3G transceiver, a 4G transceiver, and / or a 5G transceiver. Furthermore, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0135] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0136] The fourth aspect of this embodiment provides a storage medium that stores instructions containing the computer vision-based automatic empty bucket inventory and warehousing verification method described in the first aspect of the embodiment. That is, the storage medium stores instructions that, when the instructions are run on a computer, execute the computer vision-based automatic empty bucket inventory and warehousing verification method described in the first aspect of the embodiment.

[0137] The storage medium refers to a carrier for storing data, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0138] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0139] The fifth aspect of this embodiment provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the computer vision-based automatic empty bucket inventory and warehousing verification method as described in the first aspect of this embodiment. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0140] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A computer vision-based empty barrel automatic inventory and warehousing verification method, characterized in that, include: Capture images of empty buckets inside the delivery station; The empty bucket image is processed by region segmentation to separate the reflective area and the non-reflective area of ​​the bucket body in the empty bucket image. Based on the reflective area and the non-reflective area of ​​the barrel, the reflection compensation amount of each reflective pixel in the reflective area is calculated. Based on the reflection compensation amount of each reflective pixel, reflection compensation processing is performed on each reflective pixel in the empty bucket image to obtain a de-reflective image. Adaptive edge enhancement processing is performed on the de-reflected image to obtain an enhanced captured image; Edge segmentation is performed on the enhanced images to obtain the edge images of each empty bucket in the empty bucket images; Brand label recognition is performed on each edge image to obtain the brand information of each empty bucket; Based on the edge image and brand information of each empty bucket, the quantity of empty buckets of different brands in the delivery station is determined, so as to verify the empty buckets upon entry into the warehouse according to the quantity of empty buckets of different brands.

2. The method of claim 1, wherein, The image of the empty bucket is subjected to region segmentation processing to separate the reflective areas and the non-reflective areas of the bucket body in the image, including: Calculate the chromaticity mapping vector of each pixel in the empty barrel image, and use the chromaticity mapping vector of each pixel to map the RGB values ​​of each pixel in the empty barrel image to the chromaticity unit sphere to obtain the mapped image. The mapped image is subjected to an exponential transformation to obtain the transformed image; Clustering is performed on each pixel in the transformed image to obtain two pixel clusters; Obtain the light source chromaticity vector and calculate the Euclidean distance between the cluster center of two pixel clusters and the light source chromaticity vector; Based on the Euclidean distance between the cluster centers of the two pixel clusters and the chromaticity vector of the light source, the two pixel clusters are divided into a reflective cluster and a non-reflective cluster. Based on the reflective cluster and the non-reflective cluster, the reflective area and the non-reflective area of ​​the barrel body are segmented from the image of the empty barrel.

3. The method of claim 1, wherein, Based on the reflective area and the non-reflective area of ​​the barrel, the reflectivity compensation amount for each reflective pixel in the reflective area is calculated, including: For any reflective pixel in the reflective area, the RGB value of the reflective pixel is mapped onto a chromaticity unit sphere to obtain the chromaticity mapping vector of the reflective pixel. Based on the chromaticity mapping vector, the chromaticity compensation parameters of any reflective pixel are calculated. Using the chromaticity compensation parameter and chromaticity mapping vector, the first reflection suppression amount of any reflective pixel is calculated, and after traversing all reflective pixels in the reflective area, the first reflection suppression amount of each reflective pixel is obtained. Based on the non-reflective area of ​​the barrel body, the average brightness of the barrel body and the chromaticity direction component of the barrel body are calculated. The second reflectivity suppression amount is calculated using the average brightness of the barrel body and the chromaticity direction component. The reflection compensation amount for each reflective pixel in the reflective region is composed of the second reflection suppression amount and the first reflection suppression amount for each reflective pixel.

4. The method of claim 3, wherein, Based on the chromaticity mapping vector, the chromaticity compensation parameters for any reflective pixel are calculated, including: Obtain the light source chromaticity vector, and based on the light source chromaticity vector and the chromaticity mapping vector of any reflective pixel, calculate the projection coefficient of the pixel color of any reflective pixel in the direction of the light source; Using the projection coefficient, the light source chromaticity vector, and the chromaticity mapping vector, the projection vector of any reflective pixel on a plane orthogonal to the light source direction is calculated; From the chromaticity mapping vector, select the maximum and minimum mapping values; Calculate the difference between the maximum and minimum mapped values, and use the ratio between the maximum and minimum mapped values ​​as the first intermediate chromaticity parameter; Calculate the magnitude of the projection vector and obtain the product between the magnitude and the difference. The ratio between the projection coefficient and the product is used as the second chromaticity intermediate parameter; Summing the first chromaticity intermediate parameter and the second chromaticity intermediate parameter yields the initial chromaticity compensation parameter; The product of the initial chromaticity compensation parameter and the difference is calculated to obtain the third intermediate chromaticity parameter; The difference between the maximum mapping value and the third chromaticity intermediate parameter is used as the chromaticity compensation parameter.

5. The method of claim 3, wherein, The chromaticity mapping vector includes the chromaticity mapping values ​​of the R channel, the G channel, and the B channel. Using the chromaticity compensation parameters and the chromaticity mapping vector, the first reflection suppression amount for any reflective pixel is calculated, including: The chromaticity mapping values ​​of the R channel, G channel, and B channel are calculated, and the difference between them and the chromaticity compensation parameter is used to obtain the channel suppression amount of the R channel, G channel, and B channel, respectively. The first reflectivity suppression amount is formed by using the channel suppression amounts of the R channel, G channel, and B channel; The second reflectivity suppression amount is calculated using the average brightness and chromaticity direction components of the barrel body, including: The second reflectivity suppression amount is obtained by multiplying the average brightness of the barrel body with the chromaticity direction component. Accordingly, based on the reflection compensation amount of each reflective pixel, reflection compensation processing is performed on each reflective pixel in the empty bucket image, which includes: For any reflective pixel, the second reflection suppression amount and the first reflection suppression amount of the reflective pixel are weighted and summed to obtain the reflection-compensated pixel value of the reflective pixel. After all reflective pixels in the empty bucket image have been polled, the de-reflective image is obtained.

6. The method of claim 1, wherein, Adaptive edge enhancement processing is performed on the de-reflected image to obtain an enhanced captured image, including: Based on the de-reflected image, the average pixel energy of the de-reflected image is calculated; Obtain the maximum brightness value, minimum brightness value, and average brightness value of the de-reflected image; The edge enhancement adaptive threshold is calculated based on the average pixel energy, the maximum brightness value, the minimum brightness value, and the average brightness value. Based on the edge enhancement adaptive threshold, the low grayscale threshold and the high grayscale threshold are calculated; Using the low grayscale threshold and the high grayscale threshold, the de-reflected image is divided into three grayscale histograms; Establish initial probability density functions for three gray-level histograms, and then modify the three initial probability density functions to obtain three modified probability density functions; The cumulative distribution functions of the three modified probability density functions are calculated to obtain the enhancement functions of the three gray-level histograms; Obtain the maximum grayscale value in the de-reflected image; For any pixel in the de-reflective image, edge enhancement is performed on the pixel based on the maximum gray value and the enhancement function corresponding to the gray histogram to which the pixel belongs, to obtain the enhanced pixel. After all pixels in the de-reflective image have been polled, the enhanced pixels are used to form an enhanced image.

7. The method of claim 6, wherein, The edge enhancement adaptive threshold is calculated based on the average pixel energy, the maximum brightness value, the minimum brightness value, and the average brightness value, including: The edge enhancement adaptive threshold is calculated using the following formula; ； wherein denotes the edge-enhanced adaptive threshold, denotes the average luminance value, denote the maximum and minimum luminance values, respectively, denotes the average pixel energy; Accordingly, based on the edge enhancement adaptive threshold, the low grayscale threshold and high grayscale threshold are calculated, including: The low grayscale threshold and the high grayscale threshold are calculated according to the following formula; ； In the formula, These represent the high grayscale threshold and the low grayscale threshold, respectively. This represents the compensation coefficient.

8. The method according to claim 6, characterized in that, The three initial probability density functions are modified to obtain three modified probability density functions, including: For any initial probability density function, obtain the maximum probability value, minimum probability value, and correction coefficient; Calculate the probability difference between the maximum and minimum probability values; Based on the probability difference, the maximum probability value, the minimum probability value, and the correction coefficient, the initial probability density function is modified to obtain the modified probability density function corresponding to the initial probability density function.

9. A computer vision-based automatic empty bucket counting and warehousing verification system, characterized in that, include: The acquisition unit is used to acquire images of empty buckets within the delivery station. An image segmentation unit is used to perform region segmentation processing on the empty bucket image to segment out the reflective area and the non-reflective area of ​​the bucket body in the empty bucket image. The reflective compensation unit is used to calculate the reflective compensation amount of each reflective pixel in the reflective area based on the reflective area and the non-reflective area of ​​the barrel body. The reflection compensation unit is also used to perform reflection compensation processing on each reflection pixel in the empty bucket image based on the reflection compensation amount of each reflection pixel to obtain a de-reflected image. An edge enhancement unit is used to perform adaptive edge enhancement processing on the de-reflected image to obtain an enhanced captured image; The edge segmentation unit is used to perform edge segmentation on the enhanced captured image to obtain the edge image of each empty bucket in the empty bucket captured image; The label recognition unit is used to identify brand labels on each edge image to obtain brand information for each empty bucket. The counting unit is used to determine the number of empty buckets of different brands in the distribution station based on the edge image and brand information of each empty bucket, so as to verify the empty buckets entering the warehouse according to the number of empty buckets of different brands.

10. A computer program product containing instructions, characterized in that, When the instruction is executed on the computer, it causes the computer to perform the computer vision-based automatic empty bucket counting and warehousing verification method as described in any one of claims 1 to 8.

Images