Article packaging equipment control method and device, electronic equipment and readable medium

Abstract

The embodiment of the invention discloses an article packaging equipment control method and device, electronic equipment and a readable medium. A specific embodiment of the method comprises the following steps: carrying out nonlinear transformation processing on an image of an article to be packaged; performing graying processing on the transformed image of the object to be packaged; carrying out binarization processing on the grayed image of the article to be packaged; generating a feature map of the to-be-packaged article; obtaining point cloud data of a to-be-packaged article; performing coordinate transformation processing on the point cloud data of the to-be-packaged article; generating a point cloud feature map of the to-be-packaged article; generating a combined to-be-packaged article feature map according to the to-be-packaged article feature map and the to-be-packaged article point cloud feature map; inputting the combined to-be-packaged article feature map into a to-be-packaged article defect detection model to obtain a to-be-packaged article defect detection result; and in response to determining that the defect detection result of the to-be-packaged article represents that the to-be-packaged article has no defect, controlling the article packaging equipment to package the to-be-packaged article. According to the embodiment, waste of article packaging resources can be reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and particularly to a method and apparatus for controlling article packaging equipment, an electronic device, and a readable medium. Background Art

[0002] The quality of an article is the lifeline of the article and runs through every link of the article production process. In particular, as the last barrier to ensure the quality of an article, it is of great significance to do a good job in the inspection of the article before it is packaged. Currently, when performing a packaging operation on an article to be packaged, the commonly used method is to collect a planar image of the article to be packaged through an ordinary image acquisition device, detect surface defects of the article to be packaged based on the collected planar image, and perform packaging processing on the article to be packaged according to the detection result.

[0003] However, when performing a packaging operation on an article to be packaged in the above manner, the following technical problems often exist:

[0004] Detecting surface defects of the article to be packaged only through the collected planar image results in a relatively low probability of detecting defective articles to be packaged (such as opaque empty bottled or canned articles), that is, fewer defective articles to be packaged are detected, thus causing waste of article packaging resources when packaging defective articles to be packaged.

[0005] Continuing, in the process of adopting technical solutions to solve the above technical problems, the inventors found that there are relatively high requirements for the brightness and contrast of the image when detecting defects of the article to be packaged. A conventional technical solution to meet the requirements for the brightness and contrast of the image when detecting defects of the article to be packaged can be to perform histogram equalization processing on the image of the article to be packaged to enhance the brightness and contrast of the image. However, this solution will cause a large increase in background noise, a large loss of image detail information, a large decrease in information entropy, and an uneven distribution of image gray levels. Therefore, considering the requirements for the brightness and contrast of the image when detecting defects of the article to be packaged and the requirements for reducing background noise, reducing the loss of image detail information, increasing image information entropy, and evenly distributing image gray levels, the inventors decided to divide the original image into two sub-images (an image below the pixel mean and an image above the pixel mean) according to the pixel mean of the image, perform re-gray distribution on different sub-images using different cumulative density functions, and perform logarithmic transformation processing on each sub-image to increase low-gray pixels, reduce high-gray pixels, highlight the dark area details of the uneven illumination image, reduce the brightness of the high-light area, and at the same time meet the requirements for the brightness and contrast of the image when detecting defects of the article to be packaged and the requirements for reducing background noise, reducing the loss of image detail information, increasing image information entropy, and evenly distributing image gray levels.

[0006] The above information disclosed in this background art section is only used to enhance the understanding of the background of the inventive concept. Therefore, it may include information that does not form the prior art known to ordinary skilled artisans in the relevant field of the country. Summary of the Invention

[0007] This disclosure section is used to introduce the concepts in a brief form, which will be described in detail in the following detailed implementation section. This disclosure section is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0008] Some embodiments of the present disclosure propose a method, device, electronic device, and computer-readable medium for controlling an article packaging device to solve one or more of the technical problems mentioned in the above background art section.

[0009] In a first aspect, some embodiments of the present disclosure provide a method for controlling an article packaging device, the method comprising: acquiring an image of an article to be packaged; performing a non-linear transformation process on the image of the article to be packaged to obtain a non-linearly transformed image of the article to be packaged as a transformed image of the article to be packaged; performing a grayscale process on the transformed image of the article to be packaged to obtain a grayscale image of the transformed image of the article to be packaged as a grayscale image of the article to be packaged; performing a binarization process on the grayscale image of the article to be packaged to obtain a binarized grayscale image of the article to be packaged as a binarized image of the article to be packaged; generating a feature map of the article to be packaged based on the binarized image of the article to be packaged; acquiring point cloud data of the article to be packaged; performing a coordinate transformation process on the point cloud data of the article to be packaged to obtain a point cloud image of the article to be packaged; generating a point cloud feature map of the article to be packaged based on the point cloud image of the article to be packaged; generating a combined feature map of the article to be packaged based on the feature map of the article to be packaged and the point cloud feature map of the article to be packaged; inputting the combined feature map of the article to be packaged into a pre-trained defect detection model for the article to be packaged to obtain a defect detection result for the article to be packaged; and in response to determining that the defect detection result for the article to be packaged indicates that the article to be packaged is defect-free, controlling an associated article packaging device to perform a packaging process on the article to be packaged corresponding to the defect detection result for the article to be packaged.

[0010] Second aspect, some embodiments of the present disclosure provide a control device for an article packaging device. The device includes: a first acquisition unit configured to acquire an image of an article to be packaged; a transformation processing unit configured to perform non-linear transformation processing on the above-mentioned image of the article to be packaged to obtain a non-linearly transformed image of the article to be packaged as a transformed image of the article to be packaged; a grayscale processing unit configured to perform grayscale processing on the above-mentioned transformed image of the article to be packaged to obtain a grayscale processed transformed image of the article to be packaged as a grayscale image of the article to be packaged; a binarization processing unit configured to perform binarization processing on the above-mentioned grayscale image of the article to be packaged to obtain a binarized grayscale image of the article to be packaged as a binarized image of the article to be packaged; a first generation unit configured to generate a feature map of the article to be packaged according to the above-mentioned binarized image of the article to be packaged; a second acquisition unit configured to acquire point cloud data of the article to be packaged; a coordinate transformation processing unit configured to perform coordinate transformation processing on the above-mentioned point cloud data of the article to be packaged to obtain a point cloud image of the article to be packaged; a second generation unit configured to generate a point cloud feature map of the article to be packaged according to the above-mentioned point cloud image of the article to be packaged; a third generation unit configured to generate a combined feature map of the article to be packaged according to the above-mentioned feature map of the article to be packaged and the above-mentioned point cloud feature map of the article to be packaged; an input unit configured to input the above-mentioned combined feature map of the article to be packaged into a pre-trained defect detection model of the article to be packaged to obtain a defect detection result of the article to be packaged; a control unit configured to, in response to determining that the above-mentioned defect detection result of the article to be packaged indicates that the article to be packaged has no defect, control an associated article packaging device to perform packaging processing on the article to be packaged corresponding to the above-mentioned defect detection result of the article to be packaged.

[0011] Third aspect, some embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device storing one or more programs thereon, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above first aspect.

[0012] Fourth aspect, some embodiments of the present disclosure provide a computer-readable medium storing a computer program thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above first aspect is implemented.

[0013] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the method for controlling an article packaging device according to some embodiments of the present disclosure, waste of article packaging resources can be reduced. Specifically, the reason for the waste of article packaging resources is that surface defect detection of the article to be packaged is performed only based on the collected planar image, resulting in a relatively low probability of detecting defective articles to be packaged (such as opaque empty bottled or canned articles), that is, fewer defective articles to be packaged are detected, thus causing waste of article packaging resources when packaging defective articles to be packaged. Based on this, in the method for controlling an article packaging device according to some embodiments of the present disclosure, first, an image of the article to be packaged is obtained. Thus, an image of the article to be packaged representing two-dimensional planar data of the article to be packaged can be obtained. Then, non-linear transformation processing is performed on the above-mentioned image of the article to be packaged to obtain the image of the article to be packaged after non-linear transformation processing as the transformed image of the article to be packaged. Thus, the transformed image of the article to be packaged after non-linear transformation processing can be obtained, thereby enhancing the values of low gray levels or low contrast in the image and reducing the values of high gray levels or high contrast, making the overall gray level and saturation of the image balanced. After that, gray-scale processing is performed on the above-mentioned transformed image of the article to be packaged to obtain the gray-scale processed transformed image of the article to be packaged as the gray-scale image of the article to be packaged. Binary processing is performed on the above-mentioned gray-scale image of the article to be packaged to obtain the binary processed gray-scale image of the article to be packaged as the binary image of the article to be packaged. Thus, by first performing gray-scale processing and then binary processing on the image of the article to be packaged, better separation of the foreground and background of the image of the article to be packaged is achieved. Subsequently, based on the above-mentioned binary image of the article to be packaged, a feature map of the article to be packaged is generated. Thus, a feature map of the article to be packaged representing two-dimensional planar features of the article to be packaged can be obtained. Secondly, point cloud data of the article to be packaged is obtained. Thus, point cloud data of the article to be packaged representing three-dimensional stereoscopic data of the article to be packaged can be obtained. Then, coordinate transformation processing is performed on the above-mentioned point cloud data of the article to be packaged to obtain a point cloud image of the article to be packaged. Thus, a point cloud image of the article to be packaged after two-dimensional planar coordinate transformation processing can be obtained, which can be used for fusion with the image of the article to be packaged obtained by the camera sensor. After that, based on the above-mentioned point cloud image of the article to be packaged, a point cloud feature map of the article to be packaged is generated. Thus, a point cloud feature map of the article to be packaged representing three-dimensional point cloud features of the article to be packaged can be obtained. Subsequently, based on the above-mentioned feature map of the article to be packaged and the above-mentioned point cloud feature map of the article to be packaged, a combined feature map of the article to be packaged is generated. Thus, a combined feature map of the article to be packaged can be obtained, which can be used for fusing two-dimensional planar data and three-dimensional stereoscopic data of the article to be packaged. Then, the above-mentioned combined feature map of the article to be packaged is input into a pre-trained defective article detection model for the article to be packaged to obtain a defective article detection result for the article to be packaged. Thus, a defective article detection result for the article to be packaged representing whether the article to be packaged is defective can be obtained.Finally, in response to determining that the above-mentioned defect detection result of the item to be packaged indicates that the item to be packaged has no defect, control the associated item packaging device to perform packaging processing on the item to be packaged corresponding to the above-mentioned defect detection result of the item to be packaged. Thus, packaging processing can be performed on the item to be packaged without defects. Also, because the feature map of the item to be packaged representing two-dimensional plane data and the point cloud feature map of the three-dimensional solid item to be packaged are feature-fused and used for defect identification of the item to be packaged, internal defects and surface defects of the item to be packaged can be identified, thereby overall improving the defect identification ability of the item to be packaged, increasing the probability that defective items to be packaged (such as opaque bottled or canned items) are detected, that is, more defective items to be packaged are detected. Furthermore, when packaging defective items to be packaged, waste of item packaging resources can be reduced. Description of the Drawings

[0014] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the elements and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flowchart of some embodiments of a method for controlling an item packaging device according to the present disclosure;

[0016] Figure 2 is a schematic structural diagram of some embodiments of a device for controlling an item packaging device according to the present disclosure;

[0017] Figure 3 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Description of the Embodiments

[0018] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0019] In addition, it should be noted that for the sake of convenience of description, only parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0020] It should be noted that the concepts such as "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or mutual dependence relationship of the functions performed by these devices, modules or units.

[0021] It should be noted that the modification of "one" and "multiple" mentioned in this disclosure is illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of this disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The following will detail this disclosure with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 , which shows the flow 100 of some embodiments of the control method for an article packaging device according to this disclosure. The control method for the article packaging device includes the following steps:

[0025] Step 101, obtain an image of the article to be packaged.

[0026] In some embodiments, the execution subject of the control method for the article packaging device (such as a computing device) can obtain an image of the article to be packaged from an associated image acquisition device through a wired connection method or a wireless connection method. Among them, the above-mentioned associated image acquisition device can be a device capable of acquiring an image of the target article to be packaged. For example, the above-mentioned associated image acquisition device can be an industrial camera. The above-mentioned image of the article to be packaged can be an image of the article to be packaged acquired by the above-mentioned image acquisition device. The above-mentioned target article to be packaged can be any article waiting to be packaged. The above-mentioned target article to be packaged is not specifically limited here. For example, the above-mentioned target article to be packaged can be an opaque bottled food. It should be noted that the above-mentioned wireless connection method can include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other currently known or future-developed wireless connection methods.

[0027] Step 102, perform a non-linear transformation process on the image of the article to be packaged to obtain the image of the article to be packaged after the non-linear transformation process as the transformed image of the article to be packaged.

[0028] In some embodiments, the above-mentioned execution entity may perform a non-linear transformation process on the above-mentioned image of the item to be packaged, and obtain the image of the item to be packaged after the non-linear transformation process as the transformed image of the item to be packaged. In practice, the above-mentioned execution entity may perform a non-linear transformation process on the image of the item to be packaged through the following formula, and obtain the image of the item to be packaged after the non-linear transformation process as the transformed image of the item to be packaged:

[0029]

[0030] Wherein, the above-mentioned f(x, y) represents the image of the item to be packaged. The above-mentioned g(x, y) represents the transformed image of the item to be packaged. The above-mentioned x represents the abscissa of the pixel point in the image. The above-mentioned y represents the ordinate of the pixel point in the image. The above-mentioned a, b, and C are adjustment parameters that affect the position and shape of the curve, and are all constants.

[0031] Step 103: Perform grayscale processing on the transformed image of the item to be packaged, and obtain the grayscale processed transformed image of the item to be packaged as the grayscale image of the item to be packaged.

[0032] In some embodiments, the above-mentioned execution entity may perform grayscale processing on the above-mentioned transformed image of the item to be packaged, and obtain the grayscale processed transformed image of the item to be packaged as the grayscale image of the item to be packaged. Among them, the above-mentioned grayscale processing methods may include but are not limited to: maximum value method, average value method, and weighted average method. Here, the above-mentioned grayscale processing method may be the weighted average method. In practice, the above-mentioned execution entity may use the weighted average method to perform grayscale processing on the transformed image of the item to be packaged, and obtain the grayscale processed transformed image of the item to be packaged as the grayscale image of the item to be packaged.

[0033] Optionally, before the above-mentioned grayscale processing on the above-mentioned transformed image of the item to be packaged to obtain the grayscale processed transformed image of the item to be packaged as the grayscale image of the item to be packaged, the above-mentioned execution entity may further perform the following steps:

[0034] The first step: Perform image enhancement processing on the above-mentioned transformed image of the item to be packaged, and obtain the enhanced image of the item to be packaged after the image enhancement processing as the enhanced image of the item to be packaged.

[0035] The second step: Perform noise reduction processing on the above-mentioned enhanced image of the item to be packaged, and obtain the enhanced image of the item to be packaged after the noise reduction processing as the noise-reduced image of the item to be packaged.

[0036] The third step: Determine the above-mentioned noise-reduced image of the item to be packaged as the transformed image of the item to be packaged, so as to update the above-mentioned transformed image of the item to be packaged.

[0037] In some alternative implementations of some embodiments, the above-mentioned execution entity can perform noise reduction processing on the above-mentioned enhanced image of the item to be packaged to obtain the enhanced image of the item to be packaged after noise reduction processing as the image of the item to be packaged after noise reduction through the following steps:

[0038] First step, for each pixel point in the above-mentioned enhanced image of the item to be packaged, perform the following steps:

[0039] First sub-step, generate a grayscale value area according to the above-mentioned pixel point, the first preset area length, and the first preset area height. In practice, the above-mentioned execution entity can construct a grayscale value area centered on the above-mentioned pixel point, with the first preset area length as the area length and the first preset area height as the area height. Among them, the above-mentioned first preset area length can be a preset area length. The above-mentioned first preset area length can represent the length of the grayscale value area in the horizontal direction. The above-mentioned first preset area height can be a preset area height. The above-mentioned first preset area height can represent the length of the grayscale value area in the vertical direction. For example, the above-mentioned first preset area length can be 5. The above-mentioned first preset area height can be 5.

[0040] Second sub-step, generate a grayscale boundary value according to the grayscale value corresponding to the above-mentioned pixel point and the preset grayscale threshold. In practice, the above-mentioned execution entity can determine the sum of the grayscale value corresponding to the above-mentioned pixel point and the preset grayscale threshold as the grayscale boundary value. Among them, the above-mentioned preset grayscale threshold can be a preset grayscale threshold. For example, the above-mentioned preset grayscale threshold can be 25.

[0041] Third sub-step, perform sorting processing on each grayscale value included in the above-mentioned grayscale value area to obtain a grayscale value sequence. In practice, the above-mentioned execution entity can perform ascending sorting processing on each grayscale value included in the above-mentioned grayscale value area to obtain a grayscale value sequence.

[0042] Fourth sub-step, determine the median of the above-mentioned grayscale value sequence as the grayscale median value.

[0043] Fifth sub-step, determine the grayscale value with the largest value in the above-mentioned grayscale value sequence as the maximum grayscale value.

[0044] Sixth sub-step, in response to determining that the above-mentioned grayscale boundary value is greater than the above-mentioned maximum grayscale value, determine the above-mentioned grayscale median value as the grayscale value corresponding to the above-mentioned pixel point.

[0045] Seventh sub-step, determine the determined grayscale value corresponding to the above-mentioned pixel point as the updated grayscale value.

[0046] Eighth sub-step, in response to determining that the above-mentioned grayscale boundary value is less than or equal to the above-mentioned maximum grayscale value, determine the grayscale value corresponding to the above-mentioned pixel point as the updated grayscale value.

[0047] In the second step, the image composed of the determined updated gray values is determined as the image of the item to be packaged for noise reduction. Thus, in the process of denoising the enhanced image of the item to be packaged in the above first to second steps, it is possible to first determine whether the pixel belongs to the pixel corresponding to the defective area according to the relationship between the gray value of the pixel and the maximum gray value in the gray value area where the pixel is located. When the above pixel belongs to the pixel corresponding to the defective area, no filtering process is performed on the pixel, so that the edge feature information of the defective image can be retained, and further, the accuracy of the defective information of the item to be packaged generated can be improved, so that when packaging the defective item to be packaged, the waste of item packaging resources can be reduced.

[0048] In some optional implementation manners of some embodiments, the above execution subject may perform image enhancement processing on the above transformed image of the item to be packaged through the following steps to obtain the enhanced image of the item to be packaged after image enhancement processing:

[0049] In the first step, according to the above transformed image of the item to be packaged, the image pixel mean value information corresponding to the above transformed image of the item to be packaged is determined. In practice, the above execution subject may determine the mean value of the respective pixel values corresponding to the respective transformed item-to-be-packaged pixels in the above transformed item-to-be-packaged image as the image pixel mean value information.

[0050] In the second step, according to the above image pixel mean value information and the above transformed image of the item to be packaged, a low-pixel image of the item to be packaged and a high-pixel image of the item to be packaged are generated. In practice, the above execution subject may generate a low-pixel image of the item to be packaged and a high-pixel image of the item to be packaged through the following formula:

[0051] F1 = {f(x, y)|f(x, y) ≤ f m}.

[0052] F2 = {f(x, y)|f(x, y) > f m}.

[0053] Wherein, the above F1 represents the low-pixel image of the item to be packaged. The above F2 represents the high-pixel image of the item to be packaged. f(x, y) represents the pixel value corresponding to the pixel point with abscissa x and ordinate y in the transformed image of the item to be packaged. The above f m represents the image pixel mean value information. Here, 1 and 2 are used to distinguish the low-pixel image of the item to be packaged and the high-pixel image of the item to be packaged.

[0054] In the third step, the probability density corresponding to the above low-pixel image of the item to be packaged is determined. In practice, the above execution subject may determine the probability density corresponding to the above low-pixel image of the item to be packaged through the following formula:

[0055]

[0056] Among them, the above-mentioned k can represent the serial number corresponding to each pixel value in the image of the item to be packaged after transformation. The above-mentioned n1 can represent the total number of pixels included in the low-pixel image of the item to be packaged. The above-mentioned can represent the number of low-pixel item-to-be-packaged pixels with a pixel value of k in the low-pixel image of the item to be packaged. The above-mentioned p1(k) can represent the probability density corresponding to the low-pixel image of the item to be packaged.

[0057] Fourth step, determine the probability density corresponding to the above-mentioned high-pixel image of the item to be packaged. In practice, the above-mentioned execution entity can determine the probability density corresponding to the high-pixel image of the item to be packaged through the following formula:

[0058]

[0059] Among them, the above-mentioned L-1 can represent the serial number corresponding to the largest pixel value in the image of the item to be packaged after transformation. The above-mentioned n2 can represent the total number of pixels included in the high-pixel image of the item to be packaged. The above-mentioned can represent the number of high-pixel item-to-be-packaged pixels with a pixel value of k in the high-pixel image of the item to be packaged. p2(k) can represent the probability density corresponding to the high-pixel image of the item to be packaged.

[0060] Fifth step, determine the cumulative density function corresponding to the above-mentioned low-pixel image of the item to be packaged according to the probability density corresponding to the low-pixel image of the item to be packaged. In practice, the above-mentioned execution entity can determine the probability density corresponding to the low-pixel image of the item to be packaged through the following formula:

[0061]

[0062] Among them, the above-mentioned j can represent a loop variable. The above-mentioned c1(k) can represent the cumulative density function corresponding to the low-pixel image of the item to be packaged.

[0063] Sixth step, determine the cumulative density function corresponding to the above-mentioned high-pixel image of the item to be packaged according to the probability density corresponding to the high-pixel image of the item to be packaged. In practice, the above-mentioned execution entity can determine the cumulative density function corresponding to the high-pixel image of the item to be packaged through the following formula:

[0064]

[0065] Among them, the above-mentioned c2(k) can represent the cumulative density function corresponding to the high-pixel image of the item to be packaged.

[0066] Step 7: Use the cumulative density function corresponding to the above low-pixel item to be packaged image to perform image enhancement processing on the above low-pixel item to be packaged image, and obtain the low-pixel item to be packaged image after image enhancement processing as the enhanced low-pixel item to be packaged image. In practice, the above execution entity can perform image enhancement processing on the above low-pixel item to be packaged image through the following formula to obtain the low-pixel item to be packaged image after image enhancement processing as the enhanced low-pixel item to be packaged image:

[0067] g1 = f0 + (f m - f0)c1(f1).

[0068] Among them, the above g1 can represent the enhanced low-pixel item to be packaged image. The above f0 can be the minimum pixel value in the low-pixel item to be packaged image. The above f1 can be the pixel value of the current pixel in the low-pixel item to be packaged image. The above c1(f1) can be the cumulative density function corresponding to the low-pixel item to be packaged image.

[0069] Step 8: Use the cumulative density function corresponding to the above high-pixel item to be packaged image to perform image enhancement processing on the above high-pixel item to be packaged image, and obtain the high-pixel item to be packaged image after image enhancement processing as the enhanced high-pixel item to be packaged image. In practice, the above execution entity can perform image enhancement processing on the above high-pixel item to be packaged image through the following formula to obtain the high-pixel item to be packaged image after image enhancement processing as the enhanced high-pixel item to be packaged image:

[0070] g2 = f m+1 + (f L-1 - f m+1 )c2(f2).

[0071] Among them, the above g2 can represent the enhanced high-pixel item to be packaged image. The above f m+1 can be the minimum pixel value in the high-pixel item to be packaged image. The above f L-1 can be the maximum pixel value in the high-pixel item to be packaged image. The above f2 can be the pixel value of the current pixel in the high-pixel item to be packaged image. The above c2(f2) can be the cumulative density function corresponding to the high-pixel item to be packaged image.

[0072] Step 9: Perform logarithmic transformation processing on the above enhanced low-pixel item to be packaged image, and obtain the enhanced low-pixel item to be packaged image after logarithmic transformation processing as the transformed low-pixel item to be packaged image. Thus, through logarithmic transformation, low gray-level pixels can be improved.

[0073] Step 10: Perform logarithmic transformation on the above-enhanced high-pixel image of the item to be packaged, and obtain the enhanced high-pixel image of the item to be packaged after logarithmic transformation as the transformed high-pixel image of the item to be packaged. Thus, through logarithmic transformation, high gray-level pixels can be reduced.

[0074] Step 11: Generate an enhanced image of the item to be packaged based on the above-transformed low-pixel image of the item to be packaged and the above-transformed high-pixel image of the item to be packaged. In practice, the above execution entity can perform combined processing on the above-transformed low-pixel image of the item to be packaged and the above-transformed high-pixel image of the item to be packaged to obtain an enhanced image of the item to be packaged. Here, the way of combined processing can be splicing.

[0075] The above steps 1 to 11 and their related content are an inventive point of an embodiment of the present disclosure, which solves the technical problem of "during the process of image enhancement of an image, the background noise is enhanced greatly, a large amount of image detail information is lost, the information entropy decreases, the image gray levels are unevenly distributed, and it cannot well meet the higher requirements for the brightness and contrast of the image when defect detection is performed on the item to be packaged". The factors that lead to the above problems during the process of image enhancement of an image are usually as follows: performing histogram equalization processing on the image of the item to be packaged will result in greatly enhanced background noise, a large amount of lost image detail information, a large decrease in information entropy, uneven distribution of image gray levels, and it cannot well meet the requirements for the brightness and contrast of the image when defect detection is performed on the item to be packaged. If the above factors are solved, the effects of meeting the higher requirements for the brightness and contrast of the image when defect detection is performed on the item to be packaged and meeting the requirements of weakening background noise, reducing the loss of image detail information, increasing the image information entropy, and evenly distributing the image gray levels can be achieved. To achieve this effect, the present disclosure decides to divide the original image into two sub-images (an image below the pixel mean and an image above the pixel mean) according to the pixel mean of the image, perform re-gray distribution on different sub-images using different cumulative density functions, and perform logarithmic transformation on each sub-image to increase low gray-level pixels and reduce high gray-level pixels, so as to highlight the dark area details of the uneven illumination image, reduce the brightness of the high-light area, and at the same time meet the requirements for the brightness and contrast of the image when defect detection is performed on the item to be packaged and meet the requirements of weakening background noise, reducing the loss of image detail information, increasing the image information entropy, and evenly distributing the image gray levels.

[0076] Step 104: Perform binarization processing on the gray-scale image of the item to be packaged, and obtain the gray-scale image of the item to be packaged after binarization processing as the binarized image of the item to be packaged.

[0077] In some embodiments, the above-mentioned execution entity may perform binarization processing on the grayscale image of the item to be packaged, and obtain the grayscale image of the item to be packaged after binarization processing as the binary image of the item to be packaged. In practice, the above-mentioned execution entity may use Otsu's method to perform binarization processing on the grayscale image of the item to be packaged, and obtain the grayscale image of the item to be packaged after binarization processing as the binary image of the item to be packaged. Thus, by first performing grayscale processing on the image and then performing binarization processing, better separation of the foreground and background can be achieved.

[0078] Step 105: Generate a feature map of the item to be packaged according to the binary image of the item to be packaged.

[0079] In some embodiments, according to the above-mentioned binary image of the item to be packaged, the above-mentioned execution entity may generate a feature map of the item to be packaged. In practice, the above-mentioned execution entity may use a preset feature extraction module to extract features from the binary image of the item to be packaged, and obtain a feature map of the item to be packaged. Among them, the above-mentioned preset feature extraction module may be a module that is preset to be able to extract features from the binary image of the item to be packaged.

[0080] Step 106: Obtain the point cloud data of the item to be packaged.

[0081] In some embodiments, the above-mentioned execution entity may obtain the point cloud data of the item to be packaged. In practice, the above-mentioned execution entity may obtain the point cloud data of the item to be packaged from a point cloud data acquisition device of the item to be packaged through a wired connection method or a wireless connection method. Among them, the above-mentioned point cloud data acquisition device of the item to be packaged may be a device capable of collecting point cloud data of the target item to be packaged. For example, the above-mentioned point cloud data acquisition device of the item to be packaged may be a lidar set on a related item packaging device. The above-mentioned point cloud data of the item to be packaged may be the point cloud data of the item to be packaged collected by the above-mentioned point cloud data acquisition device of the item to be packaged. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other currently known or future-developed wireless connection methods.

[0082] Step 107: Perform coordinate transformation processing on the point cloud data of the item to be packaged to obtain a point cloud image of the item to be packaged.

[0083] In some embodiments, the above-mentioned execution entity may perform coordinate transformation processing on the point cloud data of the item to be packaged to obtain a point cloud image of the item to be packaged. In practice, the above-mentioned execution entity may use a preset coordinate transformation function to perform coordinate transformation processing on the point cloud data of the item to be packaged to obtain a point cloud image of the item to be packaged. Among them, the above-mentioned preset coordinate transformation function may be a function that can perform two-dimensional transformation on the point cloud data of the item to be packaged.

[0084] Optionally, before performing the coordinate transformation processing on the point cloud data of the item to be packaged to obtain a point cloud image of the item to be packaged, the above-mentioned execution entity may further perform the following steps:

[0085] First step, perform filtering processing on the point cloud data of the item to be packaged to obtain the filtered point cloud data of the item to be packaged as the filtered point cloud data of the item to be packaged. Among them, the above-mentioned filtering processing may include but is not limited to: mean filtering processing, Gaussian filtering processing, clipping filtering processing, median filtering processing, and weighted average filtering processing. Here, the above-mentioned filtering processing may be clipping filtering processing. In practice, the above-mentioned execution entity may perform clipping filtering processing on the point cloud data of the item to be packaged to obtain the filtered point cloud data of the item to be packaged as the filtered point cloud data of the item to be packaged.

[0086] Second step, perform data compression processing on the filtered point cloud data of the item to be packaged to obtain the compressed filtered point cloud data of the item to be packaged as the compressed point cloud data of the item to be packaged. In practice, the above-mentioned execution entity may use the curvature sampling method to perform data compression processing on the filtered point cloud data of the item to be packaged to obtain the compressed filtered point cloud data of the item to be packaged as the compressed point cloud data of the item to be packaged. Thus, the scattered point cloud can be thinned and compressed, effectively retaining the surface feature data of the object, and at the same time, the data processing efficiency can be improved.

[0087] Third step, determine the compressed point cloud data of the item to be packaged as the point cloud data of the item to be packaged to update the point cloud data of the item to be packaged.

[0088] Step 108, generate a point cloud feature map of the item to be packaged according to the point cloud image of the item to be packaged.

[0089] In some embodiments, according to the above-mentioned point cloud image of the item to be packaged, the above-mentioned execution entity may generate a point cloud feature map of the item to be packaged.

[0090] In practice, the above-mentioned execution entity may use a preset feature extraction module to perform feature extraction on the point cloud image of the item to be packaged to obtain a point cloud feature map of the item to be packaged. Among them, the above-mentioned preset feature extraction module may be a module that can perform feature extraction on the point cloud image of the item to be packaged.

[0091] Step 109: Generate a combined feature map of the item to be packaged based on the feature map of the item to be packaged and the point cloud feature map of the item to be packaged.

[0092] In some embodiments, based on the above-mentioned feature map of the item to be packaged and the above-mentioned point cloud feature map of the item to be packaged, the executing entity may generate a combined feature map of the item to be packaged.

[0093] In practice, the executing entity may perform a combination process on the feature map of the item to be packaged and the point cloud feature map of the item to be packaged to obtain a feature map of the item to be packaged. Here, the combination process may be adding or multiplying the corresponding elements of the two feature maps.

[0094] Step 110: Input the combined feature map of the item to be packaged into a pre-trained defect detection model for the item to be packaged to obtain a defect detection result for the item to be packaged.

[0095] In some embodiments, the executing entity may input the above-mentioned combined feature map of the item to be packaged into a pre-trained defect detection model for the item to be packaged to obtain a defect detection result for the item to be packaged. Among them, the above-mentioned defect detection model for the item to be packaged may be a neural network model that takes the combined feature map of the item to be packaged as input and the defect detection result for the item to be packaged as output. The above-mentioned defect detection result for the item to be packaged may indicate that the item to be packaged is defective or non-defective. Specifically, the above-mentioned defect detection result for the item to be packaged may include, but is not limited to, any one of the following: the item to be packaged is an empty bottle and the item to be packaged is non-defective. The above-mentioned defect detection model for the item to be packaged may include: a feature fusion layer, a weight feature extraction layer, and a detection head layer. The above-mentioned feature fusion layer may be a network layer that fuses the shallow position features and high-level semantic features of the image. The above-mentioned weight feature extraction layer may be a network layer that uses a channel attention module and a spatial attention module to extract channel features and spatial features from the image of the item to be packaged (assigning different weights according to the importance of different channel features and different spatial features). The above-mentioned detection head layer may be a network layer that can detect the position information and defect category information of the defect of the item to be packaged corresponding to the image of the item to be packaged. The above-mentioned position information may be the coordinate information of the defect corresponding to the above-mentioned item to be packaged. The above-mentioned defect category information may be that the item to be packaged is empty.

[0096] In some optional implementation manners of some embodiments, the executing entity may input the above-mentioned combined feature map of the item to be packaged into a pre-trained defect detection model for the item to be packaged through the following steps to obtain a defect detection result for the item to be packaged:

[0097] First step: Input the above-mentioned combined feature map of the item to be packaged into the above-mentioned feature fusion layer to obtain a fused feature map.

[0098] In the second step, input the above-mentioned fused feature map into the above-mentioned weight feature extraction layer to obtain a weight feature map.

[0099] In the third step, input the above-mentioned weight feature map into the above-mentioned detection head layer to obtain the detection result of the defects of the item to be packaged.

[0100] In some optional implementation manners of some embodiments, the above-mentioned detection model for the defects of the item to be packaged can be trained in the following manner:

[0101] In the first step, obtain a sample set. Among them, the samples in the above-mentioned sample set include the combined sample feature maps of the items to be packaged, and the corresponding detection results of the defects of the sample items to be packaged. Among them, the above-mentioned detection results of the defects of the sample items to be packaged can be the sample labels corresponding to the combined sample feature maps of the items to be packaged. It should be noted that the execution entity for training the above-mentioned detection model for the defects of the item to be packaged can be the above-mentioned execution entity, or other computing devices.

[0102] In the second step, perform the following training steps based on the sample set:

[0103] In the first training step, input the combined sample feature maps of at least one sample in the sample set into the initial detection model for the defects of the item to be packaged respectively, to obtain the detection results of the defects of the item to be packaged corresponding to each sample in the above-mentioned at least one sample. The above-mentioned initial detection model for the defects of the item to be packaged can be an initial neural network capable of obtaining the detection results of the defects of the item to be packaged according to the combined feature maps of the item to be packaged. The above-mentioned initial neural network can be a neural network to be trained.

[0104] In the second training step, compare the detection results of the defects of the item to be packaged corresponding to each sample in the above-mentioned at least one sample with the corresponding detection results of the defects of the sample item to be packaged. Here, the comparison can be whether the detection results of the defects of the item to be packaged corresponding to each sample in the above-mentioned at least one sample are the same as the corresponding detection results of the defects of the sample item to be packaged.

[0105] In the third training step, determine whether the initial detection model for the defects of the item to be packaged reaches a preset optimization goal according to the comparison result. Among them, the above-mentioned optimization goal can be that the accuracy rate predicted by the initial detection model for the defects of the item to be packaged is greater than or equal to a preset accuracy rate threshold. The above-mentioned preset accuracy rate threshold can be a preset accuracy rate threshold. Here, the above-mentioned preset accuracy rate threshold can be 0.95.

[0106] In the fourth training step, in response to determining that the initial detection model for the defects of the item to be packaged reaches the above-mentioned optimization goal, determine the initial detection model for the defects of the item to be packaged as the trained detection model for the defects of the item to be packaged.

[0107] Optionally, the steps of training the above-mentioned defect detection model for the item to be packaged may further include:

[0108] The fifth training step, in response to determining that the initial defect detection model for the item to be packaged does not meet the above optimization goal, adjusting the network parameters of the initial defect detection model for the item to be packaged, and using the unused samples to form a sample set, using the adjusted initial defect detection model for the item to be packaged as the initial defect detection model for the item to be packaged, and executing the above training steps again. As an example, the backpropagation algorithm (BP algorithm) and the gradient descent method (such as the mini-batch gradient descent algorithm) can be used to adjust the network parameters of the above initial defect detection model for the item to be packaged.

[0109] Step 111, in response to determining that the defect detection result of the item to be packaged indicates that the item to be packaged has no defect, controlling the associated item packaging device to perform packaging processing on the item to be packaged corresponding to the defect detection result of the item to be packaged.

[0110] In some embodiments, in response to determining that the defect detection result of the above-mentioned item to be packaged indicates that the item to be packaged has no defect, the above-mentioned execution entity may control the associated item packaging device to perform packaging processing on the item to be packaged corresponding to the defect detection result of the item to be packaged. Among them, the above-mentioned associated item packaging device may be a device capable of performing item packaging on the item to be packaged. For example, the above-mentioned associated item packaging device may be an intelligent packaging robot and an intelligent packaging robotic arm. In practice, in response to determining that the defect detection result of the above-mentioned item to be packaged indicates that the item to be packaged has no defect, the above-mentioned execution entity may control the associated item packaging device to perform packaging processing on the item to be packaged corresponding to the defect detection result of the item to be packaged.

[0111] Optionally, in response to determining that the defect detection result of the above-mentioned item to be packaged indicates that the item to be packaged has a defect, controlling the associated sound playing device to play a defect prompt message for the item to be packaged. Among them, the above-mentioned associated sound playing device may be a device capable of playing sound. For example, the above-mentioned associated sound playing device may be an amplifier or a speaker. The above-mentioned defect prompt message for the item to be packaged may be a message indicating that the item to be packaged has a packaging defect. For example, the above-mentioned defect prompt message for the item to be packaged may be "The bottle of this item to be packaged is empty. Please remove this item to be packaged from this packaging area."

[0112] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the method for controlling an article packaging device according to some embodiments of the present disclosure, waste of article packaging resources can be reduced. Specifically, the reason for the waste of article packaging resources is that surface defect detection of the article to be packaged is only performed based on the collected planar image, resulting in a relatively low probability of detecting defective articles to be packaged (such as opaque empty bottled or canned articles), that is, fewer defective articles to be packaged are detected. As a result, when packaging defective articles to be packaged, waste of article packaging resources is caused. Based on this, in the method for controlling an article packaging device according to some embodiments of the present disclosure, first, an image of the article to be packaged is obtained. Thus, an image of the article to be packaged representing two-dimensional planar data of the article to be packaged can be obtained. Then, non-linear transformation processing is performed on the above-mentioned image of the article to be packaged to obtain the image of the article to be packaged after non-linear transformation processing as the transformed image of the article to be packaged. Thus, the transformed image of the article to be packaged after non-linear transformation processing can be obtained, thereby enhancing the values of low gray levels or low contrast in the image and reducing the values of high gray levels or high contrast, making the overall gray level and saturation of the image balanced. After that, gray-scale processing is performed on the above-mentioned transformed image of the article to be packaged to obtain the transformed image of the article to be packaged after gray-scale processing as the gray-scale image of the article to be packaged. Binary processing is performed on the above-mentioned gray-scale image of the article to be packaged to obtain the gray-scale image of the article to be packaged after binary processing as the binary image of the article to be packaged. Thus, by first performing gray-scale processing on the image of the article to be packaged and then performing binary processing, better separation of the foreground and background of the image of the article to be packaged is achieved. Subsequently, according to the above-mentioned binary image of the article to be packaged, a feature map of the article to be packaged is generated. Thus, a feature map of the article to be packaged representing two-dimensional planar features of the article to be packaged can be obtained. Secondly, point cloud data of the article to be packaged is obtained. Thus, point cloud data of the article to be packaged representing three-dimensional stereoscopic data of the article to be packaged can be obtained. Then, coordinate transformation processing is performed on the above-mentioned point cloud data of the article to be packaged to obtain a point cloud image of the article to be packaged. Thus, a point cloud image of the article to be packaged after two-dimensional planar coordinate transformation processing can be obtained, which can be used for fusion with the image of the article to be packaged obtained by the camera sensor. After that, according to the above-mentioned point cloud image of the article to be packaged, a point cloud feature map of the article to be packaged is generated. Thus, a point cloud feature map of the article to be packaged representing three-dimensional point cloud features of the article to be packaged can be obtained. Subsequently, according to the above-mentioned feature map of the article to be packaged and the above-mentioned point cloud feature map of the article to be packaged, a combined feature map of the article to be packaged is generated. Thus, a combined feature map of the article to be packaged can be obtained, which can be used for fusing two-dimensional planar data of the article to be packaged and three-dimensional stereoscopic data of the article to be packaged. Then, the above-mentioned combined feature map of the article to be packaged is input into a pre-trained defective article detection model for the article to be packaged to obtain a defective article detection result for the article to be packaged. Thus, a defective article detection result for the article to be packaged representing whether the article to be packaged is defective can be obtained.Finally, in response to determining that the above-mentioned defect detection result of the item to be packaged indicates that the item to be packaged has no defect, control the associated item packaging device to perform packaging processing on the item to be packaged corresponding to the above-mentioned defect detection result of the item to be packaged. Thus, the item to be packaged without defects can be packaged. Also, because the feature map of the item to be packaged representing two-dimensional plane data and the point cloud feature map of the three-dimensional solid item to be packaged are feature-fused and used for defect identification of the item to be packaged, internal defects and surface defects of the item to be packaged can be identified, thereby improving the defect identification ability of the item to be packaged as a whole, increasing the probability that defective items to be packaged (such as opaque bottled or canned items) are detected, that is, more defective items to be packaged are detected. Furthermore, when packaging defective items to be packaged, waste of item packaging resources can be reduced.

[0113] Further referring to Figure 2 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a control device for an item packaging device. These device embodiments correspond to Figure 1 the method embodiments shown, and the device can be specifically applied to various electronic devices.

[0114] As Figure 2As shown in the figure, the control device 200 of the article packaging equipment in some embodiments includes: a first acquisition unit 201, a transformation processing unit 202, a grayscale processing unit 203, a binarization processing unit 204, a first generation unit 205, a second acquisition unit 206, a coordinate transformation processing unit 207, a second generation unit 208, a third generation unit 209, an input unit 210, and a control unit 211. Among them, the first acquisition unit 201 is configured to acquire an image of the article to be packaged; the transformation processing unit 202 is configured to perform a non-linear transformation processing on the above-mentioned image of the article to be packaged to obtain the image of the article to be packaged after the non-linear transformation processing as the transformed image of the article to be packaged; the grayscale processing unit 203 is configured to perform a grayscale processing on the above-mentioned transformed image of the article to be packaged to obtain the grayscale processed transformed image of the article to be packaged as the grayscale image of the article to be packaged; the binarization processing unit 204 is configured to perform a binarization processing on the above-mentioned grayscale image of the article to be packaged to obtain the binarized grayscale image of the article to be packaged as the binarized image of the article to be packaged; the first generation unit 205 is configured to generate a feature map of the article to be packaged according to the above-mentioned binarized image of the article to be packaged; the second acquisition unit 206 is configured to acquire the point cloud data of the article to be packaged; the coordinate transformation processing unit 207 is configured to perform a coordinate transformation processing on the above-mentioned point cloud data of the article to be packaged to obtain a point cloud image of the article to be packaged; the second generation unit 208 is configured to generate a point cloud feature map of the article to be packaged according to the above-mentioned point cloud image of the article to be packaged; the third generation unit 209 is configured to generate a combined feature map of the article to be packaged according to the above-mentioned feature map of the article to be packaged and the above-mentioned point cloud feature map of the article to be packaged; the input unit 210 is configured to input the above-mentioned combined feature map of the article to be packaged into a pre-trained defect detection model of the article to be packaged to obtain a defect detection result of the article to be packaged; the control unit 211 is configured to control the associated article packaging equipment to perform packaging processing on the article to be packaged corresponding to the defect detection result of the article to be packaged in response to determining that the defect detection result of the article to be packaged indicates that the article to be packaged has no defect.

[0115] It can be understood that the various units described in the device 200 correspond to the respective steps in the method described with reference to Figure 1 Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 200 and the units included therein, and will not be repeated here.

[0116] Next, with reference to Figure 3 , which shows a schematic structural diagram of an electronic device (such as a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0117] AsFigure 3 As shown, the electronic device 300 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 301, which may perform various appropriate actions and processes according to a program stored in the read-only memory (ROM) 302 or a program loaded from the storage device 308 into the random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0118] Generally, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 3 the electronic device 300 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had. Figure 3 Each block shown in may represent one device or, as needed, multiple devices.

[0119] Specifically, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such some embodiments, the computer program may be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above functions defined in the methods of some embodiments of the present disclosure are executed.

[0120] It should be noted that the computer-readable media described in some embodiments of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0121] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0122] The above computer-readable medium may be included in the above electronic device; or it may exist independently and not be assembled into the electronic device. The above computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device is caused to: obtain an image of an item to be packaged; perform a non-linear transformation process on the image of the item to be packaged to obtain a non-linearly transformed image of the item to be packaged as a transformed image of the item to be packaged; perform a grayscale process on the transformed image of the item to be packaged to obtain a grayscale-processed transformed image of the item to be packaged as a grayscale image of the item to be packaged; perform a binarization process on the grayscale image of the item to be packaged to obtain a binarized grayscale image of the item to be packaged as a binarized image of the item to be packaged; generate a feature map of the item to be packaged according to the binarized image of the item to be packaged; obtain point cloud data of the item to be packaged; perform a coordinate transformation process on the point cloud data of the item to be packaged to obtain a point cloud image of the item to be packaged; generate a point cloud feature map of the item to be packaged according to the point cloud image of the item to be packaged; generate a combined feature map of the item to be packaged according to the feature map of the item to be packaged and the point cloud feature map of the item to be packaged; input the combined feature map of the item to be packaged into a pre-trained defect detection model of the item to be packaged to obtain a defect detection result of the item to be packaged; and in response to determining that the defect detection result of the item to be packaged indicates that the item to be packaged is defect-free, control an associated item packaging device to perform packaging processing on the item to be packaged corresponding to the defect detection result of the item to be packaged.

[0123] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0125] The units described in some embodiments of the present disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes a first acquisition unit, a transformation processing unit, a grayscale processing unit, a binarization processing unit, a first generation unit, a second acquisition unit, a coordinate transformation processing unit, a second generation unit, a third generation unit, an input unit, and a control unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the first acquisition unit can also be described as "the unit for acquiring the image of the item to be packaged".

[0126] The functions described above can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0127] The above description is only some preferred embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, technical solutions formed by mutually replacing the above features with (but not limited to) technical features having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A control method for an article packaging device, comprising: Obtaining an image of the article to be packaged; Performing non-linear transformation processing on the image of the article to be packaged to obtain the image of the article to be packaged after non-linear transformation processing as the transformed image of the article to be packaged; Performing grayscale processing on the transformed image of the article to be packaged to obtain the grayscale processed transformed image of the article to be packaged as the grayscale image of the article to be packaged; Performing binarization processing on the grayscale image of the article to be packaged to obtain the binarized grayscale image of the article to be packaged as the binarized image of the article to be packaged; Generating a feature map of the article to be packaged according to the binarized image of the article to be packaged; Obtaining point cloud data of the article to be packaged; Performing coordinate transformation processing on the point cloud data of the article to be packaged to obtain a point cloud image of the article to be packaged; Generating a point cloud feature map of the article to be packaged according to the point cloud image of the article to be packaged; Generating a combined feature map of the article to be packaged according to the feature map of the article to be packaged and the point cloud feature map of the article to be packaged; Inputting the combined feature map of the article to be packaged into a pre-trained defect detection model for the article to be packaged to obtain a defect detection result for the article to be packaged; In response to determining that the defect detection result of the article to be packaged indicates that the article to be packaged is defect-free, controlling the associated article packaging device to perform packaging processing on the article to be packaged corresponding to the defect detection result of the article to be packaged.

2. The method according to claim 1, wherein, Before performing the coordinate transformation processing on the point cloud data of the article to be packaged to obtain a point cloud image of the article to be packaged, the method further includes: Performing filtering processing on the point cloud data of the article to be packaged to obtain the filtered point cloud data of the article to be packaged as the filtered point cloud data of the article to be packaged; Performing data compression processing on the filtered point cloud data of the article to be packaged to obtain the data-compressed filtered point cloud data of the article to be packaged as the compressed point cloud data of the article to be packaged; Determining the compressed point cloud data of the article to be packaged as the point cloud data of the article to be packaged to update the point cloud data of the article to be packaged.

3. The method according to claim 1, wherein Before performing the grayscale processing on the transformed image of the article to be packaged to obtain the grayscale processed transformed image of the article to be packaged as the grayscale image of the article to be packaged, the method further includes: Performing image enhancement processing on the transformed image of the article to be packaged to obtain the enhanced image of the article to be packaged after image enhancement processing as the enhanced image of the article to be packaged; Performing noise reduction processing on the enhanced image of the article to be packaged to obtain the noise-reduced enhanced image of the article to be packaged as the noise-reduced image of the article to be packaged; Determining the noise-reduced image of the article to be packaged as the transformed image of the article to be packaged to update the transformed image of the article to be packaged.

4. The method according to claim 1, wherein The defect detection model for the article to be packaged includes: a feature fusion layer, a weight feature extraction layer, and a detection head layer; and The step of inputting the combined feature map of the article to be packaged into a pre-trained defect detection model for the article to be packaged to obtain a defect detection result for the article to be packaged includes: Inputting the combined feature map of the article to be packaged into the feature fusion layer to obtain a fused feature map; Inputting the fused feature map into the weight feature extraction layer to obtain a weight feature map; Input the weight feature map into the detection head layer to obtain the defect detection result of the item to be packaged.

5. The method according to claim 1, wherein The defect detection model for the item to be packaged is trained in the following manner: Obtain a sample set, where the samples in the sample set include the sample combined item-to-be-packaged feature maps and the corresponding sample defect detection results of the item to be packaged for the sample combined item-to-be-packaged feature maps; Perform the following training steps based on the sample set: Input the sample combined item-to-be-packaged feature maps of at least one sample in the sample set into the initial defect detection model for the item to be packaged respectively to obtain the defect detection results of the item to be packaged corresponding to each sample in the at least one sample; Compare the defect detection results of the item to be packaged corresponding to each sample in the at least one sample with the corresponding sample defect detection results of the item to be packaged; Determine whether the initial defect detection model for the item to be packaged reaches a preset optimization goal according to the comparison result; In response to determining that the initial defect detection model for the item to be packaged reaches the optimization goal, determine the initial defect detection model for the item to be packaged as the defect detection model for the item to be packaged that has been trained.

6. The method according to claim 5, wherein The steps for training the defect detection model for the item to be packaged further include: In response to determining that the initial defect detection model for the item to be packaged does not reach the optimization goal, adjust the network parameters of the initial defect detection model for the item to be packaged, and use the unused samples to form a sample set, use the adjusted initial defect detection model for the item to be packaged as the initial defect detection model for the item to be packaged, and perform the training steps again.

7. The method according to claim 3, wherein The noise reduction processing of the enhanced item-to-be-packaged image to obtain the enhanced item-to-be-packaged image after noise reduction processing as the noise-reduced item-to-be-packaged image includes: For each pixel point in the enhanced item-to-be-packaged image, perform the following steps: Generate a gray value region according to the pixel point, the first preset region length, and the first preset region height; Generate a gray boundary value according to the gray value corresponding to the pixel point and a preset gray threshold; Perform a sorting process on the respective gray values included in the gray value region to obtain a gray value sequence; Determine the median of the gray value sequence as the gray median value; Determine the gray value with the largest value in the gray value sequence as the maximum gray value; In response to determining that the gray boundary value is greater than the maximum gray value, determine the gray median value as the gray value corresponding to the pixel point; Determine the determined gray value corresponding to the pixel point as the updated gray value; In response to determining that the gray boundary value is less than or equal to the maximum gray value, determine the gray value corresponding to the pixel point as the updated gray value; Determine the image composed of the determined respective updated gray values as the noise-reduced item-to-be-packaged image.

8. An item packaging equipment control device, including: A first acquisition unit configured to acquire an item-to-be-packaged image; A transformation processing unit configured to perform a non-linear transformation processing on the item-to-be-packaged image to obtain the item-to-be-packaged image after non-linear transformation processing as the transformed item-to-be-packaged image; A grayscale processing unit, configured to perform grayscale processing on the transformed image of the item to be packaged, and obtain the transformed image of the item to be packaged after grayscale processing as the grayscale image of the item to be packaged; A binarization processing unit, configured to perform binarization processing on the grayscale image of the item to be packaged, and obtain the grayscale image of the item to be packaged after binarization processing as the binarized image of the item to be packaged; A first generation unit, configured to generate a feature map of the item to be packaged according to the binarized image of the item to be packaged; A second acquisition unit, configured to acquire point cloud data of the item to be packaged; A coordinate transformation processing unit, configured to perform coordinate transformation processing on the point cloud data of the item to be packaged, and obtain a point cloud image of the item to be packaged; A second generation unit, configured to generate a point cloud feature map of the item to be packaged according to the point cloud image of the item to be packaged; A third generation unit, configured to generate a combined feature map of the item to be packaged according to the feature map of the item to be packaged and the point cloud feature map of the item to be packaged; An input unit, configured to input the combined feature map of the item to be packaged into a pre-trained defect detection model of the item to be packaged, and obtain a defect detection result of the item to be packaged; A control unit, configured to, in response to determining that the defect detection result of the item to be packaged indicates that the item to be packaged has no defect, control an associated item packaging device to perform packaging processing on the item to be packaged corresponding to the defect detection result of the item to be packaged.

9. An electronic device, comprising: One or more processors; A storage device, on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images