Packaging box detection method and device

Abstract

The invention provides a packaging box detection method and device, and relates to the technical field of industrial machine vision and intelligent manufacturing, and the method comprises the steps: obtaining an original image, the original image being an image obtained by photographing a to-be-detected packaging box, and the original image comprising a semitransparent seal and an identifier; obtaining an image registration reference according to the original image; based on the image registration reference, performing type detection on the original image to obtain the type of the to-be-detected packaging box; performing fuzzy processing and edge feature extraction on the original image to obtain a to-be-cut image; cutting the to-be-cut image according to the image registration reference and the type of the to-be-detected packaging box to obtain a seal feature region image and an identification region image; and inputting the seal feature area image and the identification area image into a detection model, and outputting a detection result. The problem of missing detection of the packaging box can be solved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of industrial machine vision and intelligent manufacturing, and particularly relates to a packaging box detection and a packaging box detection device. BACKGROUND

[0002] In a packaging station of a back-end test factory of products such as DRAM, a translucent qualified seal as a product qualified mark must be pasted on a packaging box. This seal not only directly affects the appearance quality of the product in vision, but also plays a key role in customer satisfaction and transportation reliability. The existence of the qualified seal provides the customer with the guarantee that the product meets the standard, and at the same time ensures the safety and integrity of the product in the transportation process.

[0003] In related technologies, in the process of seal pasting and detection in the factory, the pasting and quality inspection of the translucent seal mainly rely on manual visual inspection. Although manual operation can ensure the pasting quality of the seal to a certain extent, due to the limited human resources, the experience difference of operators and the interference of human factors, there are still missed detection cases. In addition, the mirror high light effect generated on the surface of the translucent seal makes it difficult for traditional image detection technology to achieve accurate recognition, and the consumption of computing resources is large when performing global detection.

[0004] How to effectively solve the missed detection problem with low-cost computing resources is a problem to be solved. SUMMARY

[0005] The present disclosure provides a packaging box detection method and device, which at least to some extent realizes the effective solution to the missed detection problem with low-cost computing resources.

[0006] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a packaging box detection method is provided, comprising: obtaining an original image, the original image being an image obtained by shooting a packaging box to be detected, the original image including a translucent seal and a mark; obtaining an image registration reference according to the original image; performing type detection on the original image based on the image registration reference to obtain a type of the packaging box to be detected; performing blurring processing and edge feature extraction on the original image to obtain a to-be-cut image; cutting the to-be-cut image according to the image registration reference and the type of the packaging box to be detected to obtain a seal feature region image and a mark region image; and inputting the seal feature region image and the mark region image into a detection model to output a detection result.

[0008] According to another aspect of the present disclosure, there is provided a packaging box detection device, comprising: an acquisition module configured to acquire an original image, the original image being an image obtained by photographing a packaging box to be detected, the original image comprising a semi-transparent seal and an identification; a first determination module configured to obtain an image registration reference according to the original image; a detection module configured to perform type detection on the original image based on the image registration reference to obtain a type of the packaging box to be detected; a second determination module configured to perform blurring processing and edge feature extraction on the original image to obtain a to-be-cut image; a cutting module configured to cut the to-be-cut image according to the image registration reference and the type of the packaging box to be detected to obtain a seal feature region image and an identification region image; and an output module configured to input the seal feature region image and the identification region image into a detection model and output a detection result.

[0009] According to still another aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to perform any of the above packaging box detection methods by executing the executable instructions. Illustratively, the processor can be a Graphics Processing Unit (GPU).

[0010] According to yet another aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement any of the above packaging box detection methods.

[0011] According to still another aspect of the present disclosure, there is provided a computer program product, the computer program product comprising a computer program or computer instructions, the computer program or the computer instructions being loaded and executed by a processor to cause a computer to implement any of the above packaging box detection methods.

[0012] In the embodiments of the present disclosure, an original image is obtained, the original image being an image obtained by photographing a to-be-detected packaging box, the original image including a translucent seal and an identification; an image registration reference is obtained according to the original image; a type of the to-be-detected packaging box is obtained by performing type detection on the original image based on the image registration reference; a to-be-cut image is obtained by performing blurring processing and edge feature extraction on the original image; a seal feature region image and an identification region image are obtained by cutting the to-be-cut image according to the image registration reference and the type of the to-be-detected packaging box; and a detection result is output by inputting the seal feature region image and the identification region image into a detection model. The present disclosure can eliminate specular highlight interference generated on the surface of the translucent seal by performing blurring processing and edge feature extraction on the original image. The type of the to-be-detected packaging box is obtained by performing type detection on the original image according to the image registration reference. The seal feature region image and the identification region image with smaller sizes are obtained by accurately cutting the to-be-cut image according to the type of the to-be-detected packaging box and the image registration reference. The image cutting of the present disclosure reduces the image size and eliminates noise, and the edge feature extraction enhances the target contour, which can effectively reduce the training sample amount and the computing power requirement of image recognition. In addition, the localized region detection is realized by image cutting, which prevents missed detection. The present disclosure can realize high-precision detection of the translucent seal under low computing power resources, and solve the problem of missed detection of the packaging box.

[0013] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A schematic diagram of a translucent seal is shown.

[0015] Figure 2 A schematic diagram of a packaging box detection system structure in the embodiments of the present disclosure is shown.

[0016] Figure 3 A flowchart of a packaging box detection method in the embodiments of the present disclosure is shown.

[0017] Figure 4 A schematic diagram of a Reel Box without a translucent seal in the embodiments of the present disclosure is shown.

[0018] Figure 5 A schematic diagram of a Tray Box without a translucent seal in the embodiments of the present disclosure is shown.

[0019] Figure 6 A schematic diagram of a Reel Box with a translucent seal in the embodiments of the present disclosure is shown.

[0020] Figure 7 A schematic diagram of a Tray Box with a translucent seal in the embodiments of the present disclosure is shown.

[0021] Figure 8 A schematic diagram showing the position of the minimum circumscribed rectangle in the whole image in an embodiment of the present disclosure.

[0022] Figure 9 A schematic diagram showing the original image in an embodiment of the present disclosure.

[0023] Figure 10 A schematic diagram showing the image to be cut in an embodiment of the present disclosure.

[0024] Figure 11 A region division diagram of the image to be cut in an embodiment of the present disclosure.

[0025] Figure 12 A schematic diagram showing the identification region image in an embodiment of the present disclosure.

[0026] Figure 13 A schematic diagram showing the seal feature region image in an embodiment of the present disclosure.

[0027] Figure 14 A detection result of the identification region image in an embodiment of the present disclosure.

[0028] Figure 15 A detection result of the seal feature region image in an embodiment of the present disclosure.

[0029] Figure 16 A flow chart of a packaging box detection method in an embodiment of the present disclosure.

[0030] Figure 17 A schematic diagram of a packaging box detection device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects to those skilled in the art. Features described in the description and / or shown in the figures can be combined in any suitable manner without departing from the scope of the present disclosure.

[0032] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments described herein, and together with the description serve to explain these embodiments. In the drawings: Some of the blocks in the diagrams can be functional blocks that represent functions implemented by a processor, software, or combination thereof (e.g., firmware). It will be understood that one or more functional blocks of one or more embodiments can be implemented using software, hardware, or a combination thereof.

[0033] For ease of understanding, the following first explains several terms related to this disclosure as follows.

[0034] OpenCV (Open Source Computer Vision Library) is an open-source computer vision library that provides a wide range of image processing and computer vision algorithms. OpenCV can be applied to image recognition, video analysis, machine learning, etc., and can help developers achieve efficient visual tasks.

[0035] CNN (Convolutional Neural Network) is a deep learning algorithm that can be used for image and video recognition, recommendation systems, speech recognition, etc.

[0036] YOLO (You Only Look Once) is a deep learning-based real-time object detection algorithm that can be applied to object detection tasks in images and videos.

[0037] TFLOPS (Tera Floating Point Operations Per Second) is a unit of measurement for the speed of floating-point operations processed by computer processors (such as central processing units CPU, graphics processing units GPU).

[0038] GTX is a brand name used by NVIDIA to identify its high-performance gaming graphics card series. GTX is a sub-series of the GeForce series, representing more high-end graphics card products.

[0039] RTX is a graphics card series launched by NVIDIA, representing support for Ray Tracing technology.

[0040] RGB is a color model representing the three color channels of red (Red), green (Green), and blue (Blue).

[0041] HSV is a color model representing hue (Hue), saturation (Saturation), and value (Value).

[0042] Web Service is a technology that allows different platforms and applications to communicate and exchange data over the network.

[0043] MES (Manufacturing Execution System) is a system used to manage and control manufacturing processes, which is between the enterprise's ERP system and the control system of the production site. MES is mainly responsible for optimizing production processes, improving production efficiency, ensuring product quality, and real-time monitoring of production operations.

[0044] USB (Universal Serial Bus) is a computer interface standard used to connect computers with various external devices for data transmission and power supply.

[0045] Canny algorithm is an edge detection algorithm proposed by John F. Canny in 1986.

[0046] Labelimg is an open-source image annotation tool used for image dataset annotation.

[0047] PyQt5 is a Python library for developing desktop applications, which is a Python binding of the Qt framework. Qt is a cross-platform graphical user interface (GUI) development framework.

[0048] Currently, in the process of sealing and detecting the factory's seal, the sealing work and quality inspection of the translucent seal (such as Figure 1 ) rely on manual visual inspection. Although manual operation can ensure the sealing quality of the seal to a certain extent, due to limited human resources, different operator experience, and human factors, there are missed inspections. Not only affects the smooth progress of production, but also may lead to unqualified products flowing into the market, bringing potential risks to subsequent processing.

[0049] In related technologies, in order to improve the efficiency of packaging box detection, the following detection methods are used for detection.

[0050] (1) The algorithm based on OpenCV usually relies on manually designed features for defect recognition. However, this method has the following problems: it is highly dependent on the stability of light, and changes in light and dark on the production line can easily lead to false positives; in the case of complex background interference, the recognition effect is not ideal; it is also difficult to recognize transparent objects; in addition, irregularly shaped image processing is difficult, and it cannot effectively cope with these challenges.

[0051] (2) Completely through deep learning algorithm (such as YOLO image recognition series based on CNN) to carry out image defect recognition. However, this method has the following problems: it requires ten thousand level of annotation samples and large global detection computing resources. The relationship between deep learning algorithm and required computing resources is shown in Table 1.

[0052] Table 1 Correspondence between image detection and computing resources

[0053] As can be seen from Table 1, the higher the model version, the greater the required computing resources. The computing resources required for training the model are greater than the computing resources when the model is applied.

[0054] (3) The integrated automatic packaging system (such as the LPA system) is used for packaging box recognition, which can improve the efficiency, but the cost is high, and the investment of a single workstation is between millions and tens of millions. In addition, the deployment of the system occupies a large area and has a long implementation cycle, and the deployment is difficult.

[0055] In summary, the key problems to be solved are as follows.

[0056] Solution to the computing power problem: The current market mainstream image recognition training relies on high-end graphics cards and a large number of labeled samples. However, in the case of small computer computing resources on the production line (such as only equipped with a low-end graphics card GT730 (computing power 0.69TFLOPS) launched in 2014), how to achieve effective image recognition and accurate detection through small-batch image recognition training is one of the technical difficulties of the present disclosure.

[0057] Avoiding missed detection caused by misidentification: After using the YOLOv3.6 algorithm for small-batch training, it is found in the detection process that the algorithm will misidentify the polygon object similar to the translucent seal in the production line environment as a seal. How to ensure zero missed detection (missed detection rate tends to 0) under the condition of small-batch training is an urgent difficulty that needs to be solved by the present disclosure.

[0058] Accurate detection of translucent adhesive tape seals: The translucent seal may produce reflections under different packaging box placement positions and complex lighting environments, seriously affecting the detection effect. How to accurately identify the seal area and the packaging box identification area and ensure that the first detection overjudgment rate is less than 5% is an important problem that needs to be solved by the present disclosure.

[0059] Self-adaptive detection of different sizes of packaging boxes: In the production line, there are packaging boxes of different sizes. When the fixed camera takes pictures, the position of the seal will change with the size of the packaging box. How to independently determine the type of packaging box and select the appropriate weight file for recognition while ensuring that the first detection overjudgment rate is less than 5% is a challenge that needs to be solved by the present disclosure.

[0060] Detection efficiency problem: As a key detection workstation, how to realize single packaging box detection time of not more than 1 second and ensure that the system runs stably and continuously for 7x24 hours is one of the core problems that the present disclosure focuses on.

[0061] It should be noted that the first detection error rate refers to the error rate generated by the system when detecting for the first time in an image recognition or detection task.

[0062] Based on this, the packaging box detection method provided by the embodiments of the present disclosure can be applied to the detection of products or packaging boxes. For example, it is applied to the quality control of a DRAM back-end test factory and used for detecting the translucent seal of a packaging box. The present disclosure can eliminate the specular highlight interference generated on the surface of the translucent seal by performing blurring processing and edge feature extraction on the original image. The type of the original image is detected according to the image registration reference, and the type of the packaging box to be detected is obtained. The image to be cut is accurately cut according to the type of the packaging box to be detected and the image registration reference, and the feature region image and the identification region image of the seal with a smaller size are obtained. The image cutting of the present disclosure reduces the image size, eliminates noise, and enhances the target contour through edge feature extraction, which can effectively reduce the training sample size and the computing power requirement of image recognition. In addition, the localized area detection is realized through image cutting, which prevents missed detection. The present disclosure can realize high-precision detection of the translucent seal under low computing power resources and solve the problem of missed detection of the packaging box.

[0063] The specific embodiments of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0064] Figure 2 A schematic diagram of the structure of a packaging box detection system in the embodiments of the present disclosure is shown, and the system can apply the packaging box detection method or the packaging box detection device in various embodiments of the present disclosure.

[0065] As shown in Figure 2 , the system structure can include an operation table 201, an image acquisition terminal (such as an industrial camera) 202, and at least one positioning marker 203. For example, the number of positioning markers 203 is 2, and the packaging box to be detected is located between the two positioning markers 203, and the image acquisition terminal 202 is used to shoot the packaging box to be detected.

[0066] The image acquisition terminal 202 can directly realize the detection of the packaging box, and the image acquisition terminal 202 can also transmit the original image obtained by shooting to a server (such as the server of MES) to realize the detection of the packaging box through the server. It should be noted that since the computing resources required by the present disclosure are less, appropriate electronic devices with computing power can be selected according to the needs, and the present disclosure does not make specific limitations on this.

[0067] In one embodiment, in order to improve the quality of the original image, a light source 204 is arranged above the two positioning markers 203.

[0068] In an embodiment, the operation table 201 can be replaced by a conveyor belt, and a plurality of pairs of positioning marks 203 are arranged on the surface of the conveyor belt, each pair of positioning marks 203 includes two positioning marks 203, and each detection package box is located between the two positioning marks 203. By using the conveyor belt to replace the operation table 201, the embodiment of the present disclosure can improve the automation degree of the detection of the package box, thereby improving the detection efficiency.

[0069] In an embodiment, the system structure can further include a scanning gun (not shown in the figure) through which the Box ID can be acquired. Figure 2

[0070] The quality closed-loop control of the present disclosure can be based on edge computing, and can be realized through a "perception-decision-execution" three-layer system.

[0071] For example, the quality closed-loop control of the present disclosure is realized through a perception layer, a decision layer and an execution layer. The perception layer includes the image acquisition terminal 202 composed of a high-precision industrial vision module, and the Box ID is acquired through the scanning gun. The decision layer transmits the acquired original image to a server for executing the detection of the package box to perform preprocessing and identification, thereby obtaining a detection result. Finally, the execution layer performs data interaction and verification with the MES system through the Web Service protocol, thereby completing the quality control.

[0072] The package box detection system structure disclosed in the embodiment of the present disclosure has a simple structure and low manufacturing cost.

[0073] In the embodiment of the present disclosure, a package box detection method is provided, which can be executed by any electronic device with computing processing capability.

[0074] Figure 3 A flowchart of a package box detection method in the embodiment of the present disclosure is shown in FIG. 3. Figure 3 The package box detection method provided in the embodiment of the present disclosure includes the following S301 to S306.

[0075] S301, acquiring an original image, the original image being an image obtained by shooting a detection package box, and the original image including a translucent seal and an identifier.

[0076] In the embodiment of the present disclosure, the translucent seal is a translucent qualified seal for identifying the quality of the product in the package box. For example, if a translucent seal is attached to the package box of the product, it means that the product in the package box is a qualified product after detection. For another example, if a translucent seal is attached to the package box of the product, and the translucent seal is at a preset position, it means that the product in the package box is a qualified product after detection.

[0077] ​In the embodiments of the present disclosure, the identification can be used for anti-counterfeiting and traceability, and can also be used for carrying guidance. For example, the identification can refer to the identification (such as a trademark) of the product. For another example, the identification can be a carrying instruction identification.

[0078] S302, obtaining an image registration reference according to the original image.

[0079] In the embodiments of the present disclosure, the image registration reference is a reference standard for detecting the type of the packaging box and cutting the image, which helps to improve the cutting accuracy. For example, the image registration reference includes two positioning points, and the positions of the packaging box to be detected in the original image can be accurately determined according to the two positioning points.

[0080] S303, performing type detection on the original image based on the image registration reference to obtain the type of the packaging box to be detected.

[0081] In the embodiments of the present disclosure, the type detection on the original image based on the image registration reference can improve the accuracy of the detection.

[0082] For example, a translucent seal (such as Figure 1 ) is used as a qualified identification of the product and needs to be attached to the packaging box. The types of the packaging box to be detected include two types: one is a packaging box shipped in a reel tape manner, referred to as a reel box, as shown in Figure 4 ; and the other is a packaging box shipped in a tray manner, referred to as a tray box (also referred to as a shoe box), as shown in Figure 5 . A schematic diagram of the translucent seal attached to the reel box is shown in Figure 6 , and a schematic diagram of the translucent seal attached to the tray box is shown in Figure 7 .

[0083] S304, performing blurring processing and edge feature extraction on the original image to obtain a cutting image.

[0084] In the embodiments of the present disclosure, by performing blurring processing and edge feature extraction on the original image, the mirror highlight interference generated on the surface of the translucent seal can be eliminated, the recognition degree of the translucent seal can be improved, and the probability of missing detection can be reduced.

[0085] S305, cutting the cutting image according to the image registration reference and the type of the packaging box to be detected to obtain a seal feature region image and an identification region image.

[0086] In the embodiments of the present disclosure, based on the image registration reference and the type of the packaging box to be detected, the target region (the seal feature region image and the identification region image) to be detected can be accurately cut, that is, the detection region is reduced. The reduction of the detection region can reduce the requirement for computing resources on the premise of ensuring the detection accuracy, thereby saving the production cost.

[0087] S306, input the seal feature region image and the identification region image to the detection model, and output a detection result.

[0088] The original image is subjected to blurring processing and edge feature extraction in the embodiments of the present disclosure, so that the specular highlight interference generated by the translucent seal surface can be eliminated. The type of the original image is detected according to the image registration reference, and the type of the packaging box to be detected is obtained. The to-be-cut image is accurately cut according to the type of the packaging box to be detected and the image registration reference, and the seal feature region image and the identification region image with small sizes are obtained. The image cutting of the present disclosure reduces the image size, eliminates noise, and enhances the target contour through edge feature extraction, so that the training sample amount and the computing power requirement of image recognition can be effectively reduced. In addition, the localized region detection is realized through image cutting, so that the missed detection can be prevented. The present disclosure can realize high-precision detection of the translucent seal under low computing power resources, and solve the problem of missed detection of the packaging box.

[0089] The present disclosure will be further described below through several exemplary embodiments.

[0090] In one exemplary embodiment, the original image can include a positioning mark for positioning, and the color of the positioning mark is different from the color of the packaging box to be detected. The image registration reference obtained from the original image in the packaging box detection method provided by the embodiments of the present disclosure can include the following steps A1 to A4.

[0091] Step A1, color space conversion is performed on the original image to obtain a first image.

[0092] In the embodiments of the present disclosure, the color of the positioning mark is different from the color of the packaging box to be detected, so that the positioning mark can be quickly identified from the original image.

[0093] For example, the positioning mark provided on the detection workbench (such as an operation table) has a special color (such as green) feature, so the color of the positioning mark in the original picture obtained after the industrial camera takes a picture is different from the color of the packaging box to be detected. The original image is an RGB image, and the first image is an HSV image (also known as an HSV color space model). In the embodiments of the present disclosure, color space conversion is performed on the original image, so that the robustness of color separation can be enhanced.

[0094] The shape of the positioning mark is not specifically limited in the embodiments of the present disclosure. For example, the shape of the positioning mark is L-shaped.

[0095] Exemplarily, the number of the positioning marks is 2, the shape of the positioning marks is L-shaped, and the packaging box to be detected is located between the two positioning marks. In the embodiment of the present disclosure, the L-shaped positioning marks can clearly identify the starting position and direction of the target area, avoiding the ambiguity or misjudgment caused by the traditional mark shape, and improving the positioning accuracy. In addition, the positioning marks can also position the packaging box to be detected between the two L-shaped positioning marks, preventing the packaging box to be detected from shifting.

[0096] Step A2, generating a first mask according to the first HSV threshold range corresponding to the color of the positioning mark and the first image.

[0097] In the embodiment of the present disclosure, the first mask can be a binary mask.

[0098] Exemplarily, the positioning mark is green, and the first HSV threshold range corresponding to the green color is [61, 100, 50] to [85, 255, 255]. According to the first HSV threshold range, the first image is filtered to obtain the first mask, which is used to extract the color-specific region (such as the region where the green positioning mark is located) from the original image.

[0099] In order to further improve the quality of the first mask, a closing operation (MORPH_CLOSE) and an opening operation (MORPH_OPEN) can be performed.

[0100] For example, the closing operation is used to fill the internal holes of the first mask. For another example, the opening operation is used to eliminate discrete noise points to obtain a smooth connected region.

[0101] Step A3, performing color region extraction on the original image according to the first mask to obtain a second image.

[0102] Exemplarily, the positioning mark is green, and the green region in the first mask is white (value 1) and the other regions are black (value 0). By performing a bit operation (such as an AND operation) on the original image with the first mask, the green region is accurately extracted, and the non-target region is set to zero (black).

[0103] Step A4, determining the position coordinates of the positioning mark according to the second image.

[0104] In the embodiment of the present disclosure, the position coordinates of the positioning mark are not specifically limited. For example, the position coordinates are the coordinates of the point on the outermost side of the positioning mark. For another example, the position coordinates are the coordinates of the center position of the positioning mark.

[0105] The robustness of color separation can be enhanced by color space conversion on the original image, so as to accurately determine the position coordinates of the positioning mark (i.e., the image registration reference), provide positioning criteria for the determination of the packaging box type and the cutting of the to-be-cut image, improve the accuracy of the determination of the packaging box type and the accuracy of the cutting of the to-be-cut image, and further improve the translucent seal identification degree and solve the problem of missing detection of the packaging box.

[0106] In one embodiment, the second image can include a first target extraction region, and the first target extraction region corresponds to the positioning mark, that is, the first target extraction region is a region where the positioning mark is located in the second image. The method for detecting a packaging box provided in the embodiments of the present disclosure can include the following steps A41 to A44.

[0107] In step A41, the second image is converted into a single-channel grayscale image, and a contour detection algorithm is used to identify a connected region to obtain a contour of the first target extraction region.

[0108] For example, the first target extraction region in the second image is a region, and when the second image is converted into a single-channel grayscale image, the first target extraction region becomes a region contour. The contour detection algorithm is used to identify a connected region to obtain a contour of the first target extraction region.

[0109] It should be noted that after the second image is converted into a grayscale image, the number of channels of the image is greatly reduced, and the amount of data processed is significantly reduced. This can improve the speed of image processing, and has obvious advantages especially when processing a large number of images.

[0110] It should be noted that the contour detection can help identify the shape of the target region, and even if the target image changes under different illuminations, rotations or sizes, a good detection effect can be maintained. The extraction of the contour makes the target recognition more robust, and especially in a complex background or noise environment, the misrecognition can be reduced.

[0111] In step A42, a minimum bounding rectangle of the contour of the first target extraction region is calculated to obtain the position coordinates and physical size of the minimum bounding rectangle.

[0112] In the embodiments of the present disclosure, the physical size can indicate the size of the minimum bounding rectangle of the contour of the first target extraction region, that is, the size of the minimum bounding rectangle of the contour of the first target extraction region can be determined according to the physical size. The position coordinates of the minimum bounding rectangle can indicate the position of the contour of the first target extraction region in the original image.

[0113] It should be noted that the physical size can be the length or / and width of the minimum bounding rectangle.

[0114] Step A43, determining the positioning point in the case that the position coordinates and the physical size of the minimum bounding rectangle frame satisfy the determination condition. The determination condition is that the physical size is greater than the size threshold, and the position coordinates of the minimum bounding rectangle frame satisfy the position constraint condition.

[0115] Step A44, determining the position coordinates of the positioning mark according to the positioning point.

[0116] For example, the size threshold is 0.5 cm, and the position constraint condition is that the minimum bounding rectangle frame is in the upper half of the entire image. If the physical size of the minimum bounding rectangle frame is greater than 0.5 cm, and the minimum bounding rectangle frame is in the upper half of the entire image, the diagonal vertex of the minimum bounding rectangle frame is determined as the positioning point, and the position coordinates of the positioning point are taken as the position coordinates of the positioning mark. As shown in FIG. 8B, 81 is used to indicate the minimum bounding rectangle frame, 82 is the positioning point, the minimum bounding rectangle frame is in the upper half of the entire image, and the physical size (such as the length of the minimum bounding rectangle frame) of the minimum bounding rectangle frame is greater than 0.5 cm. Figure 8

[0117] For example, the size threshold is 0.5 cm, and the position constraint condition is that the minimum bounding rectangle frame is in the upper half of the entire image. If the physical size of the minimum bounding rectangle frame is greater than 0.5 cm, and the minimum bounding rectangle frame is in the upper half of the entire image, the center of the minimum bounding rectangle frame is determined as the positioning point, and the position coordinates of the positioning point are taken as the position coordinates of the positioning mark.

[0118] The embodiment of the present disclosure can improve the image processing precision, reduce the calculation amount, enhance the robustness, and simplify the image registration process by converting the second image into a grayscale image, extracting the contour of the first target extraction region by using a contour detection algorithm, and determining the positioning point as the registration reference combined with the minimum bounding rectangle frame. The present disclosure is suitable for automatic application and can accurately and quickly detect and locate the target.

[0119] In another embodiment, the second image can include the first target extraction region, and the first target extraction region corresponds to the positioning mark, that is, the first target extraction region is the region where the positioning mark is located in the second image. The method for detecting the packaging box provided by the embodiment of the present disclosure can include the following steps: converting the second image into a single-channel grayscale image, identifying a connected region by using a contour detection algorithm, and obtaining the contour of the first target extraction region; determining the center point of the contour of the first target extraction region as the positioning point, and determining the position coordinates of the positioning mark according to the positioning point.

[0120] It should be noted that in the embodiment of the present disclosure, the shape of the positioning mark can be a rectangle.

[0121] ​By converting the second image into a grayscale image and using a contour detection algorithm to identify connected regions, the disclosed embodiment accurately extracts the contour of the target area. By using the contour center point as the positioning point, the target position can be precisely determined. The coordinates of the positioning point serve as a positioning marker, achieving efficient and accurate image positioning and registration.

[0122] In another exemplary embodiment, the packaging box detection method provided by the embodiment of the present disclosure performs type detection on the original image based on the image registration benchmark to obtain the type of the packaging box to be detected, which may include the following steps B1 and B2.

[0123] Step B1: determining a second target extraction area on the original image according to an image registration benchmark.

[0124] In the embodiment of the present disclosure, the second target extraction area is an area used to distinguish the types of packaging boxes to be detected. For example, the mark on the tray box is located in the second target extraction area, and the mark on the reel box is not located in the second target extraction area.

[0125] In the disclosed embodiment, the second target extraction region can be quickly cut out from the original image based on the image registration benchmark. For example, the right region of the upper half of the original image can be extracted as the second target region. This region can be quickly and accurately cut out based on the image registration benchmark.

[0126] For example, there are 2 positioning points, and the position coordinates of the first positioning point are ( , ), the position coordinates of the other positioning point are ( , ), based on the coordinates of the two positioning points, a rectangular area is determined. For example, two parallel lines perpendicular to the length of the original image are drawn, each passing through the two positioning points. The original image is divided into three parts, and the area between the two parallel lines is retained as the rectangular area. A directional cropping operation is performed on the area where the logo is located within the rectangular area to obtain a second target extraction area. The area of ​​the second target extraction area can be one-fourth of the area of ​​the rectangular area. In other words, the area where the logo is located is located for positioning extraction.

[0127] Step B2: determining the type of the package box to be detected based on the identifier and the second target extraction area.

[0128] In the disclosed embodiments, the specific method for determining the type of the package to be detected based on the identifier and the second target extraction area is not limited. For example, the type of the package to be detected can be determined by determining whether the identifier is located in the second target extraction area. In another example, the type of the package to be detected can be determined based on the area ratio between the identifier and the second target extraction area.

[0129] In an embodiment, the package detection method provided by the embodiments of the present disclosure can include the following steps B21 to B26.

[0130] Step B21, performing color space conversion on the second target extraction region to obtain a third image.

[0131] In the embodiments of the present disclosure, the RGB color space of the second target extraction region (i.e., the cropped sub-image) is converted into HSV color space to generate the third image. The color space conversion in the embodiments of the present disclosure can improve the stability of color domain separation.

[0132] Step B22, generating a second mask according to the second HSV threshold range corresponding to the identified color and the third image.

[0133] Step B23, performing color region extraction on the second target extraction region according to the second mask to obtain a fourth image.

[0134] For example, the identified color is red, and the second HSV threshold range includes a first red threshold range and a second red threshold range; wherein generating the second mask according to the second HSV threshold range corresponding to the identified color and the third image can include: performing OR operation on the first red threshold range and the second red threshold range to obtain a composite red threshold range; and generating the second mask according to the composite red threshold range and the third image.

[0135] For example, the first red threshold interval is defined as H∈[0, 10], S∈[100, 255], and V∈[100, 255]. The second red threshold interval is defined as H∈[160, 180], S∈[100, 255], and V∈[100, 255]. The double-interval logical OR operation is performed on the third image (HSV image) to generate the second mask. The second mask is applied to the second target extraction region through bit operation (such as AND operation) to obtain a red feature region binary image (i.e., the fourth image).

[0136] The embodiments of the present disclosure perform OR operation on the first red threshold range and the second red threshold range to obtain a composite red threshold range, and generate the second mask according to the composite red threshold range and the third image, so as to facilitate accurate color region extraction and prepare for more accurate calculation of the pixel area of the identified color.

[0137] Step B24, calculating the pixel area of the identified color according to the fourth image.

[0138] The present embodiment does not make specific limitations on how to calculate the pixel area of the mark according to the fourth image. For example, the number of the digits representing red is the pixel area of the mark, that is, the number of the digits on the fourth image is the pixel area of the mark.

[0139] It should be noted that the second mask includes two parts of digits, one part of digits (such as 1) is used to extract the fourth image, and the other part of digits (such as 0) is used to exclude the non-target area (the area outside the mark).

[0140] Step B25, calculating the ratio of the pixel area of the mark to the pixel area of the second target extraction area.

[0141] Step B26, determining the type of the packaging box to be detected according to the ratio.

[0142] For example, the present embodiment provides a packaging box detection method, which determines the type of the packaging box to be detected according to the ratio, which can include: in the case that the ratio is greater than or equal to the area ratio threshold, determining that the type of the packaging box to be detected is the first type; and in the case that the ratio is less than the area ratio threshold, determining that the type of the packaging box to be detected is the second type.

[0143] In the present embodiment, the threshold can be dynamically adjusted according to the actual application scenario and specific application experience.

[0144] For example, the pixel area of the mark is , the pixel area of the second target extraction area is , , the threshold is 10%, and the entire original image is one fourth. When / ≥ 10%, it is determined that the type of the packaging box to be detected is Tray Box; and when / < 10%, it is determined that the type of the packaging box to be detected is Reel Box.

[0145] The packaging box detection method provided by the present embodiment can accurately distinguish the type of the packaging box to be detected by calculating the pixel area ratio and comparing it with the area ratio threshold. When the ratio is greater than or equal to the threshold, it is determined to be the first type; and when the ratio is less than the threshold, it is determined to be the second type. This method is simple and efficient, and can realize the automatic and accurate classification of the packaging box type, and prepare for the subsequent accurate cutting of the image to be cut.

[0146] In another embodiment, the present embodiment provides a packaging box detection method, which determines the type of the packaging box to be detected according to the mark and the second target extraction area, which can include: determining the type of the packaging box to be detected according to the positional relationship between the mark and the second target extraction area.

[0147] For example, if the mark is located at the upper left corner of the second target extraction region, the packaging box is determined as the first type. If the mark is located at the lower right corner of the second target extraction region, the packaging box is determined as the second type.

[0148] According to the different positions of the mark, the embodiments of the present disclosure distinguish the types, and realize the accurate judgment of the packaging box type.

[0149] The embodiments of the present disclosure can realize the automatic and high-precision classification of the packaging box type through color space conversion, mask generation and pixel area ratio calculation. By accurately extracting the target region, quickly identifying the color region, and judging the target type according to the pixel area ratio, the efficiency and accuracy of target detection are significantly improved.

[0150] In another example embodiment, the packaging box detection method provided by the embodiments of the present disclosure includes the following steps C1 and C2.

[0151] Step C1, performing Gaussian filtering on the original image to obtain a blurred image.

[0152] For example, a two-dimensional Gaussian convolution kernel is used to perform the blurring processing on the original image, the convolution kernel size is (3, 3), and the standard deviation σ is 1.5. It should be noted that the embodiments of the present disclosure do not make specific limitations on the size of the convolution kernel and the value of the standard deviation.

[0153] Step C2, processing the blurred image by using an edge detection algorithm to obtain a to-be-cut image.

[0154] In the embodiments of the present disclosure, the edge detection algorithm is not specifically limited, for example, the edge detection algorithm can be a Canny multi-stage edge detection algorithm, and the double threshold parameters are set to (20, 100) to generate a binary edge feature map (i.e., the to-be-cut image).

[0155] For example, Figure 9 The original image is shown in FIG. 9, 91 is used to indicate the operation table image, 92 is used to indicate the positioning mark image, 93 is used to indicate the identification image, 94 is used to indicate the interference object image in the environment, and 95 is used to indicate the semi-transparent seal image. The original image is subjected to blurring processing and edge feature extraction to obtain a to-be-cut image, as shown in FIG. 10. Figure 10

[0156] Through the synergistic effect of Gaussian filtering and edge detection, the embodiments of the present disclosure eliminate the mirror highlight interference generated on the surface of the semi-transparent material, suppress the mirror reflection on the surface of the semi-transparent seal, output a black and white binary image that retains the edge structure of the entity, and improve the recognition accuracy of the semi-transparent seal.

[0157] ​In another exemplary embodiment, the packaging box detection method provided by the embodiment of the present disclosure may include cutting the image to be cut according to the image registration reference and the type of the packaging box to be detected to obtain the seal feature area image and the logo area image, which may include the following steps D1 and D2.

[0158] Step D1: determining cutting parameters according to the type of the package box to be inspected and the image registration reference.

[0159] Step D2: cutting the image to be cut according to the cutting parameters to obtain a seal feature area image and a logo area image.

[0160] In the disclosed embodiments, different locations on different packaging boxes require identification. Therefore, to achieve precise cutting, different cutting parameters are required for each type of box. It should be noted that different types of boxes correspond to different image registration benchmarks. These benchmarks are derived based on the actual position of the box to be inspected and the label. Using these benchmarks allows for precise cutting parameters to be determined.

[0161] For example, there are two types of packaging boxes to be inspected, one is Tray Box and the other is Reel Box. The image registration reference is the coordinates of two positioning points, where the position coordinates of the first positioning point are ( , ), the position coordinates of the other positioning point are ( , The cutting parameters of the two types of packaging boxes to be tested are shown in Table 2 below.

[0162] Table 2 Cutting parameters of the packaging box to be tested

[0163] In Table 2 above, h is the height of the image to be cut, is the horizontal coordinate of the first positioning point, is the horizontal coordinate of the second positioning point. The packaging box to be tested is located between the first and second positioning points, and the packaging box to be tested is located below the first and second positioning points. Figure 10 The position indicated by 93. The front logo is as follows Figure 6 The pattern logo in the middle of the packaging box.

[0164] For example, Figure 11 As shown, 96 is used to indicate the cutting line. When the type is Tray Box, follow h×4 / 10:h×8 / 10 and ( + )×1 / 2: Cut out the logo area image, the logo area image is as followsFigure 12 The seal feature region image is cut out according to h x 2 / 10: h x 6 / 10 and h x 2 / 10: h x 6 / 10, as shown in FIG. 6. The seal feature region image is cut out according to h x 2 / 10: h x 6 / 10 and h x 2 / 10: h x 6 / 10, as shown in FIG. 6. The seal feature region image is cut out according to h x 2 / 10: h x 6 / 10 and h x 2 / 10: h x 6 / 10, as shown in FIG. 6. Figure 13

[0165] For another example, affine transformation is performed according to the dynamic cutting coordinates, and a normalized feature region image (the seal feature region image and the identification region image) is output.

[0166] The embodiments of the present disclosure can realize accurate image cutting, extract the seal feature region image and the identification region image. The affine transformation is used to realize rotation, scaling, translation and other operations of the image, which further improves the positioning and recognition accuracy. Through effective cutting and analysis, the key features of the target region can be quickly and accurately extracted, which provides guarantee for accurate detection of the packaging box in the subsequent process and reduces the probability of missing detection of the packaging box.

[0167] In yet another example embodiment, before the seal feature region image and the identification region image are input into the detection model and a detection result is output, the packaging box detection method provided by the present disclosure can further include determining the detection model based on the type of the packaging box to be detected.

[0168] In one embodiment, the detection model corresponding to the type of the packaging box to be detected is queried according to a preset relationship correspondence table. The relationship correspondence table stores the correspondence between the type of the packaging box to be detected and the detection model.

[0169] In another embodiment, the detection model is determined based on the type of the packaging box to be detected, including: determining a weight file corresponding to the type of the packaging box to be detected; and obtaining the detection model according to a preset configuration file and the weight file.

[0170] In order to reduce the memory occupation of the electronic device by the detection model, the weight file can be determined according to the type of the packaging box to be detected, and the detection model can be obtained according to the preset configuration file and the weight file, so as to detect the packaging box by using the detection model.

[0171] In the embodiments of the present disclosure, the original image is preprocessed to obtain the seal feature region image and the identification region image, and then the seal feature region image and the identification region image are used for detection, so that the training sample amount and the computing power requirement of image recognition can be effectively reduced.

[0172] For example, the detection model is a lightweight deep learning model based on YOLO.

[0173] ​​​It should be noted that the lightweight model refers to a model with less parameters and less calculation, and less resource occupation, so that the model can run efficiently on a device with low hardware requirements. That is, in the embodiment of the present disclosure, the original image is preprocessed first, and the seal feature region image and the identification region image obtained after preprocessing can be identified using a lightweight model, thereby saving computing power while ensuring identification accuracy.

[0174] For example, the lightweight deep learning model based on YOLO uses YOLOv3.6 algorithm for carton detection.

[0175] For example, 200 photos (original images) of Tray Box and 200 photos of Reel Box are taken by using an industrial camera, and the original images are processed by using the method of S302 to S305, so that 200 identification region images and 200 seal feature region images of each type of carton are obtained. Each of the 400 images of Tray Box and Reel Box is manually labeled by using labelimg, and the model is trained based on YOLOv3.6 algorithm, and the training is performed 100 times, and the batch-size is 16 for each training. The weight files of Tray Box and Reel Box are obtained. It should be noted that the weight file belongs to a part of the detection model, and the detection model can be determined according to the weight file and the configuration file.

[0176] For example, based on YOLOv3.6, yolov3-tiny.cfg configuration file, combined with the weight file corresponding to the type of the carton to be detected, identification is performed, the confidence threshold (conf_thres) is 0.3, and the detection result is obtained, as shown in

[0177] For example, based on YOLOv3.6, yolov3-tiny.cfg configuration file, combined with the weight file corresponding to the type of the carton to be detected, identification is performed, the confidence threshold (conf_thres) is 0.3, and the detection result is obtained, as shown in Figure 14 Figure 15 For example, based on YOLOv3.6, yolov3-tiny.cfg configuration file, combined with the weight file corresponding to the type of the carton to be detected, identification is performed, the confidence threshold (conf_thres) is 0.3, and the detection result is obtained, as shown in

[0178] In the embodiment of the present disclosure, the detection model is determined according to the type of the carton to be detected, and the double-target detection is performed by using the detection model, which is beneficial to improve the detection accuracy of the carton to be detected, and realizes high-precision detection of the semi-transparent seal under low computing power resources, and solves the problem of carton missed detection.

[0179] ​In another exemplary embodiment, after acquiring the original image, the package detection method provided herein may further include obtaining a package identifier (Box ID). The Box ID can be used to identify whether the package to be detected belongs to the current site. The present embodiment does not specifically limit the form of the Box ID. For example, the Box ID can be a barcode or a QR code.

[0180] After inputting the seal feature area image and the logo area image into the detection model and outputting the detection result, the packaging box detection method provided by the present disclosure may also include: sending the packaging box identifier and the detection result to the manufacturing execution system MES, so that the MES performs verification based on the packaging box identifier and the detection result. If the detection result is passed and it is confirmed that the packaging box to be inspected belongs to the current site based on the packaging box identifier, the verification result is passed.

[0181] For example, Figure 16 As shown, the packaging box detection method provided by the present disclosure may include the following S1601 to S1604.

[0182] S1601: Determine whether the test result passes. If the test passes, proceed to S1602. If the test fails, the test result is fed back and the next package to be tested is tested. For example, if the dual-target test result (the logo area image and the seal feature area image) is "the confidence scores for the dual targets are 0.98 and 0.99, respectively," and both confidence scores are greater than 0.3, the package to be tested passes. It should be noted that if any one target fails detection, the test fails.

[0183] S1602, sending the packaging box identifier to MES.

[0184] S1603: Verify whether the package identifier is at the current site. If the verification is successful, proceed to S1604; if the verification is unsuccessful, feedback the verification result and wait for the next package to be tested.

[0185] S1604, MES passes the station and returns "Verification successful".

[0186] For example, the Box ID and test results are sent back to the MES via the Webservice protocol. The MES verifies whether the Box ID belongs to the current station. If it does and the test result is passed, the MES passes the Box ID to the current station and sends a pass message back to Packing Eye (a software name installed on an electronic device). Packing Eye displays "Verification successful." If it does not belong or the test result is failed, Packing Eye prompts an error message.

[0187] The embodiments of the present disclosure realize automatic return and real-time verification of the detection result of the packaging box by integrating the Webservice protocol, thereby improving the automation level and information flow efficiency of the production line. The system can automatically detect and return the Box ID and its detection result to the MES. After comparison and verification by the MES, it is determined whether to pass the packaging box to the next station. Through the real-time feedback mechanism, the Packing Eye timely displays "verification success" or error prompt, effectively avoiding the flow of unqualified products into the subsequent process, while ensuring production quality and improving efficiency.

[0188] The embodiments of the present disclosure return the Box ID and its detection result to the MES. After comparison and verification by the MES, it is determined whether to pass the packaging box to the next station. Through the real-time feedback mechanism, unqualified products are effectively prevented from flowing into the subsequent process, while ensuring production quality and improving efficiency.

[0189] It should be noted that the embodiments of the present disclosure can realize single packaging box detection time of not more than 1 second, and ensure that the system runs stably and continuously for 7x24 hours.

[0190] The present disclosure will be further described below through a specific embodiment.

[0191] In the exemplary embodiments, the packaging box detection process is as follows: (1) Deploy the packaging box detection hardware.

[0192] Operation table: adopt a workbench with anti-static design (for example, the workbench is equipped with ESD (electrostatic discharge) grounding function), the size is LxWxH=120x80x160cm³, 2 positioning marks (such as green L-shaped corner marks) are pasted at a distance of about 40 cm from the lower edge of the desktop of the workbench, 30 cm apart.

[0193] Imaging assembly: install an automatic focusing industrial camera (800 million pixels, maximum resolution 1920x1200, frame rate 30fps).

[0194] Illumination system: 15W (W) LED strip light source, located on the operation table, 30 cm from the desktop height.

[0195] (2) Reference positioning calibration.

[0196] Fix the camera on the support of the operation table, adjust the angle of the camera to ensure that the two kinds of packaging boxes and the positioning marks are completely located in the picture, and link the camera and the computer through USB.

[0197] (3) Model training.

[0198] It should be noted that the model training has been described in the foregoing, and will not be repeated here. The labeling tool is Labelimg, and the labeling types of the two types of packaging boxes are Tray Box: tray_box, seal and Reel Box: reel_box, seal. The model training platform can be NVIDIA GT730 (CUDA9.2), wherein NVIDIA GT730 is an NVIDIA graphics card for low-end graphics processing and some computing tasks, and CUDA9.2 is a parallel computing architecture provided by NVIDIA. The model training base network is Darknet-53 backbone, that is, Darknet-53 is used as the backbone network. The model training parameters: based on YOLOv3-tiny.cfg, modify epoch=100, batch-size=16 for training, and obtain the trained reel_box.pt and Tray_box.pt weight files, respectively.

[0199] (4) Packaging box detection and systematization.

[0200] Trigger: connect the code scanning gun and the computer through the USB, and trigger the photographing, image preprocessing and image detection by scanning the packaging box identifier (Box ID) by the code scanning gun.

[0201] Image preprocessing: preprocess the original picture according to S301 to S305 described above to obtain the type of the packaging box to be detected, the seal feature region image and the identification region image.

[0202] Image detection: according to the type of the packaging box to be detected, call the weight file obtained by training the corresponding image, use the YOLOv3.6 algorithm to detect the target region (seal feature region image and identification region image), and transmit the Box ID and detection result to the MES.

[0203] Interaction with the MES system: after the MES obtains the Box ID and the detection result, the MES checks whether the Box ID belongs to the station, if it belongs to the station and the detection result is passed, the MES passes the station of the Box ID, and returns the passing information to the Packing Eye, and the Packing Eye displays “check success”; if it does not belong to the station or the detection result is not passed, the Packing Eye will prompt an error message.

[0204] Interface integration: develop the software interface based on PyQt5 and integrate the above steps, and complete the installation and deployment of the production line computer.

[0205] Based on the same inventive concept, the present disclosure also provides a packaging box detection device, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0206] Figure 17 A schematic diagram of a packaging box detection device according to an embodiment of the present disclosure is shown. Figure 17 As shown, the packaging box detection device includes: an acquisition module 171, a first determination module 172, a detection module 173, a second determination module 174, a cutting module 175, and an output module 176. The acquisition module 171 can be used to acquire an original image, which is an image obtained by photographing the packaging box to be detected, and the original image includes a translucent seal and a logo; the first determination module 172 can be used to obtain an image registration reference based on the original image; the detection module 173 can be used to perform type detection on the original image based on the image registration reference to obtain the type of the packaging box to be detected; the second determination module 174 can be used to perform blurring processing and edge feature extraction on the original image to obtain an image to be cut; the cutting module 175 can be used to cut the image to be cut based on the image registration reference and the type of the packaging box to be detected to obtain a seal feature area image and a logo area image; and the output module 176 can be used to input the seal feature area image and the logo area image into the detection model and output the detection results.

[0207] In one embodiment, the original image includes a positioning mark for positioning, and the color of the positioning mark is different from the color of the packaging box to be detected; the first determination module 172 can also be used to perform color space conversion on the original image to obtain a first image; generate a first mask based on the first HSV threshold range corresponding to the color of the positioning mark and the first image; perform color area extraction on the original image based on the first mask to obtain a second image; determine the position coordinates of the positioning mark based on the second image.

[0208] In one embodiment, the second image includes a first target extraction area, and the first target extraction area corresponds to the positioning mark; the first determination module 172 can also be used to convert the second image into a single-channel grayscale image, and identify the connected area through the contour detection algorithm to obtain the contour of the first target extraction area; calculate the minimum circumscribed rectangular box of the contour of the first target extraction area to obtain the position coordinates and physical size of the minimum circumscribed rectangular box; determine the positioning point when the position coordinates and physical size of the minimum circumscribed rectangular box meet the judgment condition; determine the position coordinates of the positioning mark according to the positioning point; wherein the judgment condition is that the physical size is greater than the size threshold and the position coordinates of the minimum circumscribed rectangular box meet the position constraint condition.

[0209] In an embodiment, the detection module 173 can be further configured to determine a second target extraction region on the original image according to the image registration reference; and determine the type of the to-be-detected packaging box according to the mark and the second target extraction region.

[0210] In an embodiment, the detection module 173 can be further configured to perform color space conversion on the second target extraction region to obtain a third image; generate a second mask according to a second HSV threshold range corresponding to the color of the mark and the third image; perform color region extraction on the second target extraction region according to the second mask to obtain a fourth image; calculate the pixel area of the mark according to the fourth image; calculate a ratio of the pixel area of the mark to the pixel area of the second target extraction region; and determine the type of the to-be-detected packaging box according to the ratio.

[0211] In an embodiment, the color of the mark is red, and the second HSV threshold range includes a first red threshold range and a second red threshold range; the detection module 173 can be further configured to perform OR operation on the first red threshold range and the second red threshold range to obtain a composite red threshold range; and generate the second mask according to the composite red threshold range and the third image.

[0212] In an embodiment, the detection module 173 can be further configured to determine that the type of the to-be-detected packaging box is a first type when the ratio is greater than or equal to an area ratio threshold; and determine that the type of the to-be-detected packaging box is a second type when the ratio is less than the area ratio threshold.

[0213] In an embodiment, the second determination module 174 can be further configured to perform Gaussian filtering on the original image to obtain a blurred image; and process the blurred image by using an edge detection algorithm to obtain the to-be-cut image.

[0214] In an embodiment, the cutting module 175 can be further configured to determine a cutting parameter according to the type of the to-be-detected packaging box and the image registration reference; and cut the to-be-cut image according to the cutting parameter to obtain the seal feature region image and the mark region image.

[0215] In an embodiment, before inputting the seal feature region image and the mark region image into the detection model and outputting a detection result, the output module 176 can be further configured to determine the detection model based on the type of the to-be-detected packaging box.

[0216] In an embodiment, the output module 176 can be further configured to determine a weight file corresponding to the type of the to-be-detected packaging box; and obtain the detection model according to a preset configuration file and the weight file.

[0217] In an embodiment, the detection model is a lightweight deep learning model based on YOLO.

[0218] In an embodiment, the number of positioning marks is 2, the shape of the positioning marks is L-shaped, and the packaging box to be detected is located between the two positioning marks.

[0219] In an embodiment, after obtaining the original image, the obtaining module 171 can also be configured to obtain a packaging box identifier; wherein, after inputting the seal feature region image and the identification region image into the detection model and outputting the detection result, the output module 176 can also be configured to send the packaging box identifier and the detection result to a manufacturing execution system (MES) to enable the MES to perform verification according to the packaging box identifier and the detection result, and in the case that the detection result is passed and the packaging box to be detected is confirmed to belong to the current site according to the packaging box identifier, the verification result is passed.

[0220] The packaging box detection device disclosed in the embodiments of the present disclosure can effectively reduce the training sample amount and the computing power requirement of image recognition. In addition, the localized region detection is realized through image cutting, which prevents missed detection. The present disclosure can realize high-precision detection of a semi-transparent seal under low computing power resources, and solve the problem of missed detection of packaging boxes.

[0221] According to still another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the steps of any of the above packaging box detection methods via execution of the executable instructions. Illustratively, the processor can be a graphics processing unit (GPU).

[0222] According to still another aspect of the present disclosure, a computer-readable storage medium is provided, having a computer program stored thereon, which is executed by a processor to implement the steps of any of the above packaging box detection methods.

[0223] According to still another aspect of the present disclosure, a computer program product is provided, comprising a computer program or computer instructions, which are loaded and executed by a processor to enable a computer to implement any of the above packaging box detection methods.

[0224] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. In the embodiments of the present disclosure, any reference to memory, storage, database or other medium can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), disk memory, CD-ROM, and optical memory. Volatile memory can include random access memory (RAM), dynamic random access memory (DRAM), and static random access memory (SRAM). The term "computer-readable medium" includes, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and various other mediums capable of storing, containing, or carrying instruction(s) and / or data. Examples of a computer-readable medium include a floppy disk, a 3.5-inch flexible disk, a hard disk, a solid state drive, a magnetic tape, a cassette tape, an optical disk, a compact disk, a digital versatile disk, a Blu-ray disk, a memory card, a memory stick, and / or any other medium capable of storing software. The computer-readable medium can be a transitory or non-transitory computer-readable medium. The term "computer-readable medium" is also intended to include "wired" and / or "wireless" transmission media that transmit programming code. The term "computer-readable medium" is intended to also include various types of communications links and / or carriers, such as hardwired links, wireless links, communication links, and / or carrier waves that transmit programming code.

[0225] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. The disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such features that are evident from the specification and claims. The specification and examples are illustrative only and not restrictive of the disclosure. The true scope of the disclosure is set forth in the claims.

Claims

1. A packaging box detection method, characterized in that: include: Acquire an original image, where the original image is obtained by photographing the packaging box to be inspected, and the original image includes a translucent seal and a logo; Obtaining an image registration benchmark according to the original image; Based on the image registration benchmark, performing type detection on the original image to obtain the type of the packaging box to be detected; Performing fuzzy processing and edge feature extraction on the original image to obtain an image to be cut; Cutting the image to be cut according to the image registration reference and the type of the package box to be inspected to obtain a seal feature area image and a logo area image; The seal feature area image and the logo area image are input into a detection model, and a detection result is output.

2. The packaging box detection method according to claim 1, characterized in that: The original image includes a positioning mark for positioning, and the color of the positioning mark is different from the color of the packaging box to be detected; Wherein, obtaining an image registration benchmark according to the original image includes: Performing color space conversion on the original image to obtain a first image; generating a first mask according to a first HSV threshold range corresponding to the color of the positioning mark and the first image; performing color region extraction on the original image according to the first mask to obtain a second image; The position coordinates of the positioning mark are determined according to the second image.

3. The packaging box detection method according to claim 2, characterized in that: The second image includes a first target extraction area, and the first target extraction area corresponds to the positioning mark; Wherein, determining the position coordinates of the positioning mark according to the second image includes: Converting the second image into a single-channel grayscale image, and identifying connected regions using a contour detection algorithm to obtain a contour of the first target extraction region; Calculating a minimum bounding rectangle of the outline of the first target extraction area, and obtaining position coordinates and physical dimensions of the minimum bounding rectangle; When the position coordinates and physical size of the minimum circumscribed rectangular frame meet the determination conditions, determining the positioning point; Determine the position coordinates of the positioning mark according to the positioning point; The determination condition is that the physical size is greater than a size threshold, and the position coordinates of the minimum circumscribed rectangular frame meet position constraint conditions.

4. The packaging box detection method according to claim 2, characterized in that: The performing type detection on the original image based on the image registration reference to obtain the type of the package box to be detected includes: Determining a second target extraction area on the original image according to the image registration benchmark; The type of the packaging box to be detected is determined according to the identification and the second target extraction area.

5. The packaging box detection method according to claim 4, characterized in that: The determining the type of the package box to be detected according to the identifier and the second target extraction area includes: performing color space conversion on the second target extraction area to obtain a third image; generating a second mask based on a second HSV threshold range corresponding to the color of the marker and the third image; performing color region extraction on the second target extraction region according to the second mask to obtain a fourth image; calculating the pixel area of ​​the mark according to the fourth image; Calculating a ratio of a pixel area of ​​the marker to a pixel area of ​​the second target extraction region; The type of the packaging box to be detected is determined according to the ratio.

6. The packaging box detection method according to claim 5, characterized in that: The color of the mark is red, and the second HSV threshold range includes a first red threshold range and a second red threshold range; The step of generating a second mask based on a second HSV threshold range corresponding to the color of the marker and the third image includes: Performing an OR operation on the first red threshold range and the second red threshold range to obtain a composite red threshold range; A second mask is generated based on the composite red threshold range and the third image.

7. The packaging box detection method according to claim 5, characterized in that: Determining the type of the package box to be detected according to the ratio includes: When the ratio is greater than or equal to the area ratio threshold, determining that the type of the package box to be detected is the first type; When the ratio is smaller than the area ratio threshold, it is determined that the type of the package box to be detected is the second type.

8. The packaging box detection method according to claim 1, characterized in that: The fuzzy processing and edge feature extraction of the original image to obtain the image to be cut includes: Performing Gaussian filtering on the original image to obtain a blurred image; The blurred image is processed using an edge detection algorithm to obtain the image to be cut.

9. The packaging box detection method according to claim 1, characterized in that: According to the image registration reference and the type of the package box to be inspected, the image to be cut is cut to obtain a seal feature area image and a logo area image, including: Determining cutting parameters according to the type of the packaging box to be inspected and the image registration reference; The image to be cut is cut according to the cutting parameters to obtain the seal feature area image and the logo area image.

10. The packaging box detection method according to claim 1, characterized in that: Before inputting the seal feature area image and the logo area image into the detection model and outputting the detection result, the method further includes: A detection model is determined based on the type of the package box to be detected.

11. The packaging box detection method according to claim 10, characterized in that: The determining of the detection model based on the type of the package box to be detected includes: Determine a weight file corresponding to the type of the package box to be detected; The detection model is obtained according to the preset configuration file and the weight file.

12. The packaging box detection method according to any one of claims 2 to 4, characterized in that: The number of the positioning marks is 2, the shape of the positioning marks is L-shaped, and the package box to be inspected is located between the two positioning marks.

13. The packaging box detection method according to claim 11, characterized in that: After acquiring the original image, the method further includes: Get the packaging box identifier; After inputting the seal feature area image and the logo area image into the detection model and outputting the detection result, the method further includes: The packaging box identifier and the detection result are sent to the manufacturing execution system MES, so that the MES performs verification based on the packaging box identifier and the detection result. If the detection result is passed and it is confirmed that the packaging box to be inspected belongs to the current site based on the packaging box identifier, the verification result is passed.

14. A packaging box detection device, characterized in that: include: An acquisition module, configured to acquire an original image, wherein the original image is obtained by photographing the packaging box to be inspected, and the original image includes a translucent seal and a logo; A first determining module, configured to obtain an image registration reference according to the original image; A detection module, configured to perform type detection on the original image based on the image registration reference to obtain the type of the package box to be detected; A second determining module is used to perform fuzzy processing and edge feature extraction on the original image to obtain an image to be cut; A cutting module, configured to cut the image to be cut according to the image registration reference and the type of the package to be inspected, to obtain a seal feature area image and a logo area image; The output module is used to input the seal feature area image and the logo area image into the detection model and output the detection result.

Images