Packaging box detection method and device

Abstract

The disclosure provides a packaging box detection method and device, and relates to the technical field of industrial machine vision and intelligent manufacturing. The method comprises the following steps: obtaining an original image, wherein the original image is an image obtained by shooting a packaging box to be detected, and the original image comprises a translucent seal and an identification; obtaining an image registration reference according to the original image; performing type detection on the original image based on the image registration reference to obtain the type of the packaging box to be detected; performing blurring processing and edge feature extraction on the original image to obtain a to-be-cut image; cutting the to-be-cut image according to the image registration reference and the type of the packaging box to be detected to obtain a seal feature region image and an identification region image; and inputting the seal feature region image and the identification region image into a detection model to output a detection result. The disclosure can solve the problem of packaging box missed detection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of industrial machine vision and intelligent manufacturing, and particularly relates to a packaging box detection and a packaging box detection device. BACKGROUND

[0002] In a packaging station of a back-end test factory of products such as DRAM, a translucent qualified seal as a product qualified mark must be pasted on a packaging box. This seal not only directly affects the appearance quality of the product in vision, but also plays a key role in customer satisfaction and transportation reliability. The existence of the qualified seal provides the customer with the guarantee that the product meets the standard, and at the same time ensures the safety and integrity of the product in the transportation process.

[0003] In related technologies, in the process of seal pasting and detection in the factory, the pasting and quality inspection of the translucent seal mainly rely on manual visual inspection. Although manual operation can ensure the pasting quality of the seal to a certain extent, due to the limited human resources, the experience difference of operators and the interference of human factors, there are still missed detection cases. In addition, the mirror high light effect generated on the surface of the translucent seal makes it difficult for traditional image detection technology to achieve accurate recognition, and the consumption of computing resources is large when performing global detection.

[0004] How to effectively solve the missed detection problem with low-cost computing resources is a problem to be solved. SUMMARY

[0005] The present disclosure provides a packaging box detection method and device, which at least to some extent realizes the effective solution to the missed detection problem with low-cost computing resources.

[0006] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a packaging box detection method is provided, comprising: obtaining an original image, the original image being an image obtained by shooting a packaging box to be detected, the original image including a translucent seal and a mark; obtaining an image registration reference according to the original image; performing type detection on the original image based on the image registration reference to obtain a type of the packaging box to be detected; performing blurring processing and edge feature extraction on the original image to obtain a to-be-cut image; cutting the to-be-cut image according to the image registration reference and the type of the packaging box to be detected to obtain a seal feature region image and a mark region image; and inputting the seal feature region image and the mark region image into a detection model to output a detection result.

[0008] According to another aspect of the present disclosure, there is provided a packaging box detection device, comprising: an acquisition module configured to acquire an original image, the original image being an image obtained by photographing a packaging box to be detected, the original image comprising a semi-transparent seal and an identification; a first determination module configured to obtain an image registration reference according to the original image; a detection module configured to perform type detection on the original image based on the image registration reference to obtain a type of the packaging box to be detected; a second determination module configured to perform blurring processing and edge feature extraction on the original image to obtain a to-be-cut image; a cutting module configured to cut the to-be-cut image according to the image registration reference and the type of the packaging box to be detected to obtain a seal feature region image and an identification region image; and an output module configured to input the seal feature region image and the identification region image into a detection model and output a detection result.

[0009] According to still another aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to perform any of the above packaging box detection methods by executing the executable instructions. Illustratively, the processor can be a Graphics Processing Unit (GPU).

[0010] According to yet another aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement any of the above packaging box detection methods.

[0011] According to still another aspect of the present disclosure, there is provided a computer program product, the computer program product comprising a computer program or computer instructions, the computer program or the computer instructions being loaded and executed by a processor to cause a computer to implement any of the above packaging box detection methods.

[0012] In the embodiments of the present disclosure, an original image is obtained, the original image being an image obtained by photographing a to-be-detected packaging box, the original image including a translucent seal and an identification; an image registration reference is obtained according to the original image; a type of the to-be-detected packaging box is obtained by performing type detection on the original image based on the image registration reference; a to-be-cut image is obtained by performing blurring processing and edge feature extraction on the original image; a seal feature region image and an identification region image are obtained by cutting the to-be-cut image according to the image registration reference and the type of the to-be-detected packaging box; and a detection result is output by inputting the seal feature region image and the identification region image into a detection model. The present disclosure can eliminate specular highlight interference generated on the surface of the translucent seal by performing blurring processing and edge feature extraction on the original image. The type of the to-be-detected packaging box is obtained by performing type detection on the original image according to the image registration reference. The seal feature region image and the identification region image with smaller sizes are obtained by accurately cutting the to-be-cut image according to the type of the to-be-detected packaging box and the image registration reference. The image cutting of the present disclosure reduces the image size, eliminates noise, and enhances the target contour through edge feature extraction, which can effectively reduce the training sample amount and the computing power requirement of image recognition. In addition, the localized region detection is realized through image cutting, which prevents missed detection. The present disclosure can realize high-precision detection of the translucent seal under low computing power resources, and solve the problem of missed detection of the packaging box.

[0013] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A schematic diagram of a translucent seal is shown.

[0015] Figure 2 A schematic diagram of a packaging box detection system structure in the embodiments of the present disclosure is shown.

[0016] Figure 3 A flowchart of a packaging box detection method in the embodiments of the present disclosure is shown.

[0017] Figure 4 A schematic diagram of a Reel Box without a translucent seal in the embodiments of the present disclosure is shown.

[0018] Figure 5 A schematic diagram of a Tray Box without a translucent seal in the embodiments of the present disclosure is shown.

[0019] Figure 6 A schematic diagram of a Reel Box with a translucent seal in the embodiments of the present disclosure is shown.

[0020] Figure 7 A schematic diagram of a Tray Box with a translucent seal in the embodiments of the present disclosure is shown.

[0021] Figure 8 A schematic diagram showing the position of the minimum circumscribed rectangle in the whole image in an embodiment of the present disclosure.

[0022] Figure 9 A schematic diagram showing the original image in an embodiment of the present disclosure.

[0023] Figure 10 A schematic diagram showing the image to be cut in an embodiment of the present disclosure.

[0024] Figure 11 A region division diagram of the image to be cut in an embodiment of the present disclosure.

[0025] Figure 12 A schematic diagram showing the identification region image in an embodiment of the present disclosure.

[0026] Figure 13 A schematic diagram showing the seal feature region image in an embodiment of the present disclosure.

[0027] Figure 14 A detection result of the identification region image in an embodiment of the present disclosure.

[0028] Figure 15 A detection result of the seal feature region image in an embodiment of the present disclosure.

[0029] Figure 16 A flow chart of a packaging box detection method in an embodiment of the present disclosure.

[0030] Figure 17 A schematic diagram of a packaging box detection device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects to those skilled in the art. Features described in the description and / or shown in the figures can be combined in any suitable manner without departing from the scope of the present disclosure.

[0032] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments described herein, and together with the description serve to explain these embodiments. In the drawings: Some of the blocks in the diagrams can be functional blocks that represent functions implemented by a processor, software, or combination thereof (e.g., firmware). It will be understood that one or more functional blocks of one or more embodiments can be implemented using software, hardware, or a combination thereof.

[0033] For ease of understanding, the following first explains several terms related to this disclosure as follows.

[0034] OpenCV (Open Source Computer Vision Library) is an open-source computer vision library that provides a wide range of image processing and computer vision algorithms. OpenCV can be applied to image recognition, video analysis, machine learning, etc., and can help developers achieve efficient visual tasks.

[0035] CNN (Convolutional Neural Network) is a deep learning algorithm that can be used for image and video recognition, recommendation systems, speech recognition, etc.

[0036] YOLO (You Only Look Once) is a deep learning-based real-time object detection algorithm that can be applied to object detection tasks in images and videos.

[0037] TFLOPS (Tera Floating Point Operations Per Second) is a unit of measurement for the speed of floating-point operations processed by computer processors such as CPUs and GPUs.

[0038] GTX is a brand name used by NVIDIA to identify its high-performance gaming graphics card series. GTX is a sub-series of the GeForce series, representing more high-end graphics cards.

[0039] RTX is a graphics card series launched by NVIDIA, representing support for Ray Tracing technology.

[0040] RGB is a color model representing the three color channels of red, green, and blue.

[0041] HSV is a color model representing hue, saturation, and value.

[0042] Web Service is a technology that allows different platforms and applications to communicate and exchange data over the network.

[0043] MES (Manufacturing Execution System) is a system used to manage and control manufacturing processes, which is between the enterprise's ERP system and the control system of the production site. MES is mainly responsible for optimizing production processes, improving production efficiency, ensuring product quality, and real-time monitoring of production operations.

[0044] USB (Universal Serial Bus) is a computer interface standard used to connect computers with various external devices for data transmission and power supply.

[0045] Canny algorithm is an edge detection algorithm proposed by John F. Canny in 1986.

[0046] Labelimg is an open-source image annotation tool used for image dataset annotation.

[0047] PyQt5 is a Python library for developing desktop applications, which is a Python binding of the Qt framework. Qt is a cross-platform graphical user interface (GUI) development framework.

[0048] Currently, in the process of sealing and detecting the factory's seal, the sealing work and quality inspection of the translucent seal (such as Figure 1 ) rely on manual visual inspection. Although manual operation can ensure the sealing quality of the seal to a certain extent, due to limited human resources, different operator experience, and human factors, there are missed inspections. Not only affects the smooth progress of production, but also may lead to unqualified products flowing into the market, bringing potential risks to subsequent processing.

[0049] In related technologies, in order to improve the efficiency of packaging box detection, the following detection methods are used for detection.

[0050] (1) The algorithm based on OpenCV usually relies on manually designed features for defect recognition. However, this method has the following problems: it is highly dependent on the stability of light, and changes in light and dark on the production line can easily lead to false positives; in the case of complex background interference, the recognition effect is not ideal; it is also difficult to recognize transparent objects; in addition, irregularly shaped image processing is difficult, and it cannot effectively cope with these challenges.

[0051] (2) Completely through deep learning algorithm (such as YOLO image recognition series based on CNN) to carry out image defect recognition. However, this method has the following problems: it requires ten thousand level of annotation samples and large global detection computing resources. The relationship between deep learning algorithm and required computing resources is shown in Table 1.

[0052] Table 1 correspondence between image detection and computing resources

[0053]

[0054] As can be seen from Table 1, the higher the model version, the greater the required computing resources. The computing resources required for training the model are greater than the computing resources when the model is applied.

[0055] (3) The integrated automatic packaging system (such as the LPA system) is used for packaging box recognition, which can improve the efficiency, but the cost is high, and the investment of a single workstation is between millions and tens of millions. In addition, the deployment of the system occupies a large area and has a long implementation cycle, and the deployment is difficult.

[0056] In summary, the key problems to be solved are as follows.

[0057] Solution to the computing power problem: the current market mainstream image recognition training relies on high-end graphics cards and a large number of labeled samples. However, in the case of small computer computing resources on the production line (such as only equipped with a low-end graphics card GT730 (computing power 0.69TFLOPS) launched in 2014), how to achieve effective image recognition and accurate detection through small-batch image recognition training is one of the technical difficulties of the present disclosure.

[0058] Avoiding missed detection caused by misidentification: after using the YOLOv3.6 algorithm for small-batch training, it is found in the detection process that the algorithm will misidentify the polygon object similar to the translucent seal in the surrounding environment of the production line as a seal. How to ensure zero missed detection (missed detection rate tends to 0) under the condition of small-batch training is an important problem to be solved in the present disclosure.

[0059] Accurate detection of translucent adhesive tape seals: the translucent seal may produce reflection under different packaging box placement positions and complex lighting environments, seriously affecting the detection effect. How to accurately identify the seal area and the packaging box identification area and ensure that the first detection overjudgment rate is less than 5% is an important problem to be solved in the present disclosure.

[0060] Self-adaptive detection of different sizes of packaging boxes: in the production line, there are packaging boxes of different sizes. When the fixed camera takes a picture, the position of the seal will change with the size of the packaging box. How to independently determine the type of packaging box and select the appropriate weight file for recognition while ensuring that the first detection overjudgment rate is less than 5% is a challenge to be solved in the present disclosure.

[0061] Detection efficiency problem: as a key detection workstation, how to realize single packaging box detection time of not more than 1 second and ensure that the system runs stably and continuously for 7x24 hours is one of the core problems of the present disclosure.

[0062] It should be noted that the first detection error rate refers to the error rate generated by the system when detecting for the first time in an image recognition or detection task.

[0063] Based on this, the packaging box detection method provided by the embodiments of the present disclosure can be applied to the detection of products or packaging boxes. For example, it can be applied to quality control in a DRAM back-end test factory to detect the translucent seal of a packaging box. The present disclosure can eliminate the specular highlight interference generated on the surface of the translucent seal by blurring the original image and extracting the edge features. The type of the original image is detected according to the image registration reference, and the type of the packaging box to be detected is obtained. The image to be cut is accurately cut according to the type of the packaging box to be detected and the image registration reference, and the feature region image and the identification region image of the seal with a smaller size are obtained. The image cutting of the present disclosure reduces the image size, eliminates noise, and enhances the target contour through edge feature extraction, which can effectively reduce the training sample size and the computing power requirement of image recognition. In addition, the localized area detection is realized through image cutting to prevent missed detection. The present disclosure can realize high-precision detection of the translucent seal under low computing power resources and solve the problem of missed detection of the packaging box.

[0064] The specific embodiments of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0065] Figure 2 A schematic diagram of the structure of a packaging box detection system in the embodiments of the present disclosure is shown, and the system can apply the packaging box detection method or the packaging box detection device in various embodiments of the present disclosure.

[0066] As shown in Figure 2 , the system structure can include an operation table 201, an image acquisition terminal (such as an industrial camera) 202, and at least one positioning marker 203. For example, the number of positioning markers 203 is 2, and the packaging box to be detected is located between the two positioning markers 203, and the image acquisition terminal 202 is used to shoot the packaging box to be detected.

[0067] The image acquisition terminal 202 can directly realize packaging box detection, and the image acquisition terminal 202 can also transmit the original image obtained by shooting to a server (such as the server of MES) to realize packaging box detection through the server. It should be noted that since the computing resources required by the present disclosure are less, appropriate electronic devices with computing power can be selected according to needs, and the present disclosure does not make specific limitations on this.

[0068] In one embodiment, in order to improve the quality of the original image, a light source 204 is arranged above the two positioning markers 203.

[0069] In an embodiment, the operation table 201 can be replaced by a conveyor belt, and a plurality of pairs of positioning marks 203 are arranged on the surface of the conveyor belt, each pair of positioning marks 203 includes two positioning marks 203, and each detection package box is located between the two positioning marks 203. By using the conveyor belt to replace the operation table 201, the embodiment of the present disclosure can improve the automation degree of the detection of the package box, thereby improving the detection efficiency.

[0070] In an embodiment, the system structure can further include a scanning gun (not shown in the figure) through which the Box ID can be acquired. Figure 2

[0071] The embodiment of the present disclosure can realize quality closed-loop control through a "perception-decision-execution" three-layer system based on edge computing.

[0072] For example, the quality closed-loop control of the present disclosure is realized through a perception layer, a decision layer, and an execution layer. The perception layer includes the image acquisition terminal 202 composed of a high-precision industrial vision module, and the Box ID is acquired through the scanning gun. The decision layer transmits the acquired original image to a server for executing the detection of the package box to perform preprocessing and identification, thereby obtaining the detection result. Finally, the execution layer performs data interaction and verification with the MES system through the Web Service protocol, thereby completing the quality control.

[0073] The package box detection system structure disclosed in the embodiment of the present disclosure has a simple structure and low manufacturing cost.

[0074] In the embodiment of the present disclosure, a package box detection method is provided, which can be executed by any electronic device with computing processing capability.

[0075] Figure 3 A flowchart of a package box detection method in the embodiment of the present disclosure is shown in FIG. 3. Figure 3 The package box detection method provided in the embodiment of the present disclosure includes the following S301 to S306.

[0076] S301, acquiring an original image, the original image being an image obtained by shooting a detection package box, and the original image including a translucent seal and an identifier.

[0077] In the embodiment of the present disclosure, the translucent seal is a translucent qualified seal for identifying the quality of the product in the package box. For example, if a translucent seal is attached to the package box of a product, it means that the product in the package box is a qualified product after detection. For another example, if a translucent seal is attached to the package box of a product, and the translucent seal is at a preset position, it means that the product in the package box is a qualified product after detection.

[0078] ​In the embodiments of the present disclosure, the identification can be used for anti-counterfeiting and traceability, and can also be used for carrying guidance. For example, the identification can refer to the identification (such as a trademark) of the product. For another example, the identification can be a carrying instruction identification.

[0079] S302, obtaining an image registration reference according to the original image.

[0080] In the embodiments of the present disclosure, the image registration reference is a reference standard for detecting the type of the packaging box and cutting the image, which helps to improve the cutting accuracy. For example, the image registration reference includes two positioning points, and the positions of the packaging box to be detected in the original image can be accurately determined according to the two positioning points.

[0081] S303, performing type detection on the original image based on the image registration reference to obtain the type of the packaging box to be detected.

[0082] In the embodiments of the present disclosure, the type detection on the original image based on the image registration reference can improve the accuracy of the detection.

[0083] For example, a translucent seal (such as Figure 1 ) is used as a qualified identification of the product and needs to be attached to the packaging box. The types of the packaging box to be detected include two types: one is a packaging box shipped in a reel tape manner, referred to as a reel box, as shown in Figure 4 ; and the other is a packaging box shipped in a tray manner, referred to as a tray box (also referred to as a shoe box), as shown in Figure 5 . A schematic diagram of the translucent seal attached to the reel box is shown in Figure 6 , and a schematic diagram of the translucent seal attached to the tray box is shown in Figure 7 .

[0084] S304, performing blurring processing and edge feature extraction on the original image to obtain a cutting image.

[0085] In the embodiments of the present disclosure, by performing blurring processing and edge feature extraction on the original image, the mirror highlight interference generated on the surface of the translucent seal can be eliminated, the recognition degree of the translucent seal can be improved, and the probability of missed detection can be reduced.

[0086] S305, cutting the cutting image according to the image registration reference and the type of the packaging box to be detected to obtain a seal feature region image and an identification region image.

[0087] In the embodiments of the present disclosure, based on the image registration reference and the type of the packaging box to be detected, the target region (the seal feature region image and the identification region image) to be detected can be accurately cut, that is, the detection region is reduced. The reduction of the detection region can reduce the requirement for computing resources on the premise of ensuring the detection accuracy, thereby saving the production cost.

[0088] S306, input the seal feature region image and the identification region image to the detection model, and output a detection result.

[0089] The original image is subjected to blurring processing and edge feature extraction in the embodiment of the present disclosure, so that the specular highlight interference generated by the translucent seal surface can be eliminated. The type of the original image is detected according to the image registration reference, and the type of the packaging box to be detected is obtained. The image to be cut is accurately cut according to the type of the packaging box to be detected and the image registration reference, and the seal feature region image and the identification region image with small size are obtained. The image cutting of the present disclosure reduces the image size, eliminates noise, and enhances the target contour through edge feature extraction, so that the training sample amount and the computing power requirement of image recognition can be effectively reduced. In addition, the localized region detection is realized through image cutting, so as to prevent missed detection. The present disclosure can realize high-precision detection of the translucent seal under low computing power resources, and solve the problem of missed detection of the packaging box.

[0090] The present disclosure will be further described below through several exemplary embodiments.

[0091] In one exemplary embodiment, the original image can include a positioning mark for positioning, and the color of the positioning mark is different from the color of the packaging box to be detected. The image registration reference obtained from the original image in the packaging box detection method provided by the embodiment of the present disclosure can include the following steps A1 to A4.

[0092] Step A1, color space conversion is performed on the original image to obtain a first image.

[0093] In the embodiment of the present disclosure, the color of the positioning mark is different from the color of the packaging box to be detected, so that the positioning mark can be quickly identified from the original image.

[0094] For example, the positioning mark provided with a special color (such as green) feature is arranged on the detection workbench (such as an operation table), so that the color of the positioning mark in the original picture obtained after the industrial camera is photographed is different from the color of the packaging box to be detected. The original image is an RGB image, and the first image is an HSV image (also known as HSV color space model). In the embodiment of the present disclosure, color space conversion is performed on the original image, so that the robustness of color separation can be enhanced.

[0095] The shape of the positioning mark is not specifically limited in the embodiment of the present disclosure. For example, the shape of the positioning mark is L-shaped.

[0096] Exemplarily, the number of the positioning marks is 2, the shape of the positioning marks is L-shaped, and the packaging box to be detected is located between the two positioning marks. In the embodiment of the present disclosure, the L-shaped positioning marks can clearly identify the starting position and direction of the target area, avoiding the ambiguity or misjudgment caused by the traditional mark shape, and improving the positioning accuracy. In addition, the positioning marks can also position the packaging box to be detected between the two L-shaped positioning marks, preventing the packaging box to be detected from shifting.

[0097] Step A2, generating a first mask according to the first HSV threshold range corresponding to the color of the positioning mark and the first image.

[0098] In the embodiment of the present disclosure, the first mask can be a binary mask.

[0099] Exemplarily, the positioning mark is green, and the first HSV threshold range corresponding to the green color is [61, 100, 50] to [85, 255, 255]. According to the first HSV threshold range, the first image is filtered to obtain the first mask, which is used to extract the color-specific region (such as the region where the green positioning mark is located) from the original image.

[0100] In order to further improve the quality of the first mask, a closing operation (MORPH_CLOSE) and an opening operation (MORPH_OPEN) can be performed.

[0101] For example, the closing operation is used to fill the internal holes of the first mask. For another example, the opening operation is used to eliminate discrete noise points to obtain a smooth connected region.

[0102] Step A3, performing color region extraction on the original image according to the first mask to obtain a second image.

[0103] Exemplarily, the positioning mark is green, and the green region in the first mask is white (value 1) and the other regions are black (value 0). By performing a bit operation (such as an AND operation) on the original image with the first mask, the green region is accurately extracted, and the non-target region is set to zero (black).

[0104] Step A4, determining the position coordinates of the positioning mark according to the second image.

[0105] In the embodiment of the present disclosure, the position coordinates of the positioning mark are not specifically limited. For example, the position coordinates are the coordinates of the point on the outermost side of the positioning mark. For another example, the position coordinates are the coordinates of the center position of the positioning mark.

[0106] The robustness of color separation can be enhanced by color space conversion on the original image, so as to accurately determine the position coordinates of the positioning mark (i.e., the image registration reference), provide a positioning standard for the determination of the packaging box type and the cutting of the to-be-cut image, improve the accuracy of the determination of the packaging box type and the accuracy of the cutting of the to-be-cut image, and further improve the translucent seal identification degree and solve the problem of missed detection of the packaging box.

[0107] In one embodiment, the second image can include a first target extraction region, and the first target extraction region corresponds to the positioning mark, that is, the first target extraction region is a region where the positioning mark is located in the second image. The method for detecting a packaging box provided in the embodiments of the present disclosure can include the following steps A41 to A44.

[0108] In step A41, the second image is converted into a single-channel grayscale image, and a contour detection algorithm is used to identify a connected region to obtain a contour of the first target extraction region.

[0109] For example, the first target extraction region in the second image is a region, and when the second image is converted into a single-channel grayscale image, the first target extraction region becomes a region contour. The contour detection algorithm is used to identify a connected region to obtain a contour of the first target extraction region.

[0110] It should be noted that after the second image is converted into a grayscale image, the number of channels of the image is greatly reduced, and the amount of data processed is significantly reduced. This can improve the speed of image processing, and has obvious advantages especially when processing a large number of images.

[0111] It should be noted that the contour detection can help identify the shape of the target region, and even if the target image changes under different illuminations, rotations or sizes, a good detection effect can be maintained. The extraction of the contour makes the target recognition more robust, and especially in a complex background or noise environment, the misrecognition can be reduced.

[0112] In step A42, a minimum bounding rectangle of the contour of the first target extraction region is calculated to obtain the position coordinates and physical size of the minimum bounding rectangle.

[0113] In the embodiments of the present disclosure, the physical size can indicate the size of the minimum bounding rectangle of the contour of the first target extraction region, that is, the size of the minimum bounding rectangle of the contour of the first target extraction region can be determined according to the physical size. The position coordinates of the minimum bounding rectangle can indicate the position of the contour of the first target extraction region in the original image.

[0114] It should be noted that the physical size can be the length or / and width of the minimum bounding rectangle.

[0115] Step A43, determining the positioning point in the case that the position coordinates and the physical size of the minimum bounding rectangle frame satisfy the determination condition. The determination condition is that the physical size is greater than the size threshold, and the position coordinates of the minimum bounding rectangle frame satisfy the position constraint condition.

[0116] Step A44, determining the position coordinates of the positioning mark according to the positioning point.

[0117] For example, the size threshold is 0.5 cm, and the position constraint condition is that the minimum bounding rectangle frame is in the upper half of the whole image. If the physical size of the minimum bounding rectangle frame is greater than 0.5 cm, and the minimum bounding rectangle frame is in the upper half of the whole image, the diagonal vertex of the minimum bounding rectangle frame is determined as the positioning point, and the position coordinates of the positioning point are taken as the position coordinates of the positioning mark. As shown in FIG. 8B, 81 is used to indicate the minimum bounding rectangle frame, 82 is the positioning point, the minimum bounding rectangle frame is in the upper half of the whole image, and the physical size (such as the length of the minimum bounding rectangle frame) of the minimum bounding rectangle frame is greater than 0.5 cm. Figure 8

[0118] For example, the size threshold is 0.5 cm, and the position constraint condition is that the minimum bounding rectangle frame is in the upper half of the whole image. If the physical size of the minimum bounding rectangle frame is greater than 0.5 cm, and the minimum bounding rectangle frame is in the upper half of the whole image, the center of the minimum bounding rectangle frame is determined as the positioning point, and the position coordinates of the positioning point are taken as the position coordinates of the positioning mark.

[0119] The embodiment of the present disclosure can improve the image processing precision, reduce the calculation amount, enhance the robustness, and simplify the image registration process by converting the second image into a gray-scale image, extracting the contour of the first target extraction region by using a contour detection algorithm, and determining the positioning point as the registration reference combined with the minimum bounding rectangle frame. The present disclosure is suitable for automatic application and can accurately and quickly detect and locate the target.

[0120] In another embodiment, the second image can include the first target extraction region, and the first target extraction region corresponds to the positioning mark, that is, the first target extraction region is the region where the positioning mark is located in the second image. The method for detecting the packaging box provided by the embodiment of the present disclosure can include the following steps: converting the second image into a single-channel gray-scale image, identifying a connected region by using a contour detection algorithm, and obtaining the contour of the first target extraction region; determining the center point of the contour of the first target extraction region as the positioning point, and determining the position coordinates of the positioning mark according to the positioning point.

[0121] It should be noted that in the embodiment of the present disclosure, the shape of the positioning mark can be a rectangle.

[0122] ​This embodiment of the disclosure converts the second image into a grayscale image and uses a contour detection algorithm to identify connected regions, thereby accurately extracting the contour of the target region. By determining the center point of the contour as a positioning point, the target position can be accurately determined, and the coordinates of the positioning point can be used as positioning markers, achieving efficient and accurate image positioning and registration.

[0123] In another exemplary embodiment, the packaging box detection method provided in this disclosure performs type detection on the original image based on an image registration reference to obtain the type of the packaging box to be detected, which may include the following steps B1 and B2.

[0124] Step B1: Determine the second target extraction region on the original image based on the image registration reference.

[0125] In this embodiment of the disclosure, the second target extraction region is a region used to distinguish the type of packaging box to be detected. For example, the markings on a Tray Box are located in the second target extraction region, while the markings on a Reel Box are not located in the second target extraction region.

[0126] In this embodiment of the disclosure, a second target extraction region can be quickly cut out from the original image based on an image registration reference. For example, the right-hand region of the upper half of the original image can be extracted as the second target region. Based on the image registration reference, this region can be cut out quickly and accurately.

[0127] For example, there are two positioning points. The coordinates of the first positioning point are ( , The coordinates of the other positioning point are ( , Based on the position coordinates of two positioning points, a rectangular region is determined. For example, two parallel lines perpendicular to the length direction of the original image are drawn, each passing through one of the two positioning points, dividing the original image into three parts. The region between the two parallel lines is retained as the aforementioned rectangular region. A directional cropping operation is performed on the region containing the marker within the rectangular region to obtain the second target extraction region. The area of ​​the second target extraction region can be one-quarter of the area of ​​the rectangular region. In other words, the region containing the marker is located and extracted.

[0128] Step B2: Determine the type of packaging box to be detected based on the identifier and the second target extraction area.

[0129] In this embodiment, the specific method for determining the type of the packaging box to be detected based on the identifier and the second target extraction area is not limited. For example, the type of packaging box to be detected can be determined by whether the identifier is located in the second target extraction area. Another example is to determine the type of packaging box to be detected based on the area ratio of the identifier to the second target extraction area.

[0130] In an embodiment, the package detection method provided by the embodiments of the present disclosure can include the following steps B21 to B26.

[0131] Step B21, performing color space conversion on the second target extraction region to obtain a third image.

[0132] In the embodiments of the present disclosure, the RGB color space of the second target extraction region (i.e., the cropped sub-image) is converted into HSV color space to generate the third image. The color space conversion in the embodiments of the present disclosure can improve the stability of color domain separation.

[0133] Step B22, generating a second mask according to the second HSV threshold range corresponding to the identified color and the third image.

[0134] Step B23, performing color region extraction on the second target extraction region according to the second mask to obtain a fourth image.

[0135] For example, the identified color is red, and the second HSV threshold range includes a first red threshold range and a second red threshold range; wherein generating the second mask according to the second HSV threshold range corresponding to the identified color and the third image can include: performing OR operation on the first red threshold range and the second red threshold range to obtain a composite red threshold range; and generating the second mask according to the composite red threshold range and the third image.

[0136] For example, the first red threshold interval is defined as H∈[0, 10], S∈[100, 255], and V∈[100, 255]. The second red threshold interval is defined as H∈[160, 180], S∈[100, 255], and V∈[100, 255]. The double-interval logical OR operation is performed on the third image (HSV image) to generate the second mask. The second mask is applied to the second target extraction region through bit operation (such as AND operation) to obtain a red feature region binary image (i.e., the fourth image).

[0137] The embodiments of the present disclosure perform OR operation on the first red threshold range and the second red threshold range to obtain a composite red threshold range, and generate the second mask according to the composite red threshold range and the third image, so as to facilitate accurate color region extraction and prepare for more accurate calculation of the pixel area of the mark.

[0138] Step B24, calculating the pixel area of the mark according to the fourth image.

[0139] The present embodiment does not make specific limitations on how to calculate the pixel area of the mark according to the fourth image. For example, the number of the digits representing red is the pixel area of the mark, that is, the number of the digits on the fourth image is the pixel area of the mark.

[0140] It should be noted that the second mask includes two parts of digits, one part of digits (such as 1) is used to extract the fourth image, and the other part of digits (such as 0) is used to exclude the non-target area (the area outside the mark).

[0141] Step B25, calculating the ratio of the pixel area of the mark to the pixel area of the second target extraction area.

[0142] Step B26, determining the type of the packaging box to be detected according to the ratio.

[0143] For example, the present embodiment provides a packaging box detection method, which determines the type of the packaging box to be detected according to the ratio, which can include: in the case that the ratio is greater than or equal to the area ratio threshold, determining that the type of the packaging box to be detected is the first type; and in the case that the ratio is less than the area ratio threshold, determining that the type of the packaging box to be detected is the second type.

[0144] In the present embodiment, the threshold can be dynamically adjusted according to the actual application scenario and specific application experience.

[0145] For example, the pixel area of the mark is , the pixel area of the second target extraction area is , , the threshold is 10%, and the entire original image is one fourth. When / is greater than or equal to 10%, it is determined that the type of the packaging box to be detected is Tray Box; and when / is less than 10%, it is determined that the type of the packaging box to be detected is Reel Box.

[0146] The packaging box detection method provided by the present embodiment can accurately distinguish the type of the packaging box to be detected by calculating the pixel area ratio and comparing it with the area ratio threshold. When the ratio is greater than or equal to the threshold, it is determined to be the first type; and when the ratio is less than the threshold, it is determined to be the second type. This method is simple and efficient, and can realize the automatic and accurate classification of the packaging box type, and prepare for the subsequent accurate cutting of the image to be cut.

[0147] In another embodiment, the present embodiment provides a packaging box detection method, which determines the type of the packaging box to be detected according to the mark and the second target extraction area, which can include: determining the type of the packaging box to be detected according to the positional relationship between the mark and the second target extraction area.

[0148] For example, if the mark is located at the upper left corner of the second target extraction region, the packaging box is determined as the first type. If the mark is located at the lower right corner of the second target extraction region, the packaging box is determined as the second type.

[0149] According to the different positions of the mark, the embodiments of the present disclosure distinguish the types, and realize the accurate judgment of the packaging box type.

[0150] The embodiments of the present disclosure can realize the automatic and high-precision classification of the packaging box type through color space conversion, mask generation and pixel area ratio calculation. By accurately extracting the target region, quickly identifying the color region, and judging the target type according to the pixel area ratio, the efficiency and accuracy of target detection are significantly improved.

[0151] In another example embodiment, the packaging box detection method provided by the embodiments of the present disclosure includes the following steps C1 and C2.

[0152] Step C1, performing Gaussian filtering on the original image to obtain a blurred image.

[0153] For example, a two-dimensional Gaussian convolution kernel is used to perform the blurring processing on the original image, the convolution kernel size is (3, 3), and the standard deviation σ is 1.5. It should be noted that the embodiments of the present disclosure do not make specific limitations on the size of the convolution kernel and the value of the standard deviation.

[0154] Step C2, processing the blurred image by using an edge detection algorithm to obtain a to-be-cut image.

[0155] In the embodiments of the present disclosure, the edge detection algorithm is not specifically limited, for example, the edge detection algorithm can be a Canny multi-stage edge detection algorithm, and the double threshold parameters are set to (20, 100) to generate a binary edge feature map (i.e., the to-be-cut image).

[0156] For example, Figure 9 The original image is shown in FIG. 9, 91 is used to indicate the operation table image, 92 is used to indicate the positioning mark image, 93 is used to indicate the identification image, 94 is used to indicate the interference object image in the environment, and 95 is used to indicate the semi-transparent seal image. The original image is processed by blurring and edge feature extraction to obtain a to-be-cut image, as shown in FIG. 10. Figure 10

[0157] Through the synergistic effect of Gaussian filtering and edge detection, the embodiments of the present disclosure eliminate the mirror highlight interference generated on the surface of the semi-transparent material, suppress the mirror reflection on the surface of the semi-transparent seal, output a black and white binary image that retains the edge structure of the entity, and improve the recognition accuracy of the semi-transparent seal.

[0158] ​In yet another example embodiment, the package detection method provided by the present disclosure provides that, according to the image registration reference and the type of the package to be detected, the cutting of the to-be-cut image to obtain the seal feature region image and the identification region image can include the following steps D1 and D2.

[0159] Step D1, determining the cutting parameters according to the type of the package to be detected and the image registration reference.

[0160] Step D2, cutting the to-be-cut image according to the cutting parameters to obtain the seal feature region image and the identification region image.

[0161] In the example embodiment of the present disclosure, different types of packages need to identify different positions, and therefore, in order to achieve accurate cutting, different cutting parameters need to be selected for different types of packages. It should be noted that the image registration reference corresponding to different types of packages is different, and the image registration reference is obtained according to the actual position of the package to be detected and the identification. Using the actual image registration reference can determine the accurate cutting parameters.

[0162] For example, the type of the package to be detected has two types, one is Tray Box, and the other is Reel Box. The image registration reference is the coordinates of two positioning points, wherein the position coordinates of the first positioning point are (x1, y1), and the position coordinates of the other positioning point are (x2, y2). The cutting parameters of the two types of packages to be detected are shown in Table 2. , ,

[0163] Table 2 Cutting parameters of the package to be detected

[0164]

[0165] In the above Table 2, h is the height of the to-be-cut image, x1 is the horizontal coordinate of the first positioning point, x2 is the horizontal coordinate of the second positioning point, the package to be detected is located between the first positioning point and the second positioning point, and the package to be detected is located below the first positioning point and the second positioning point. The carrying identification is indicated in the position of 93 in FIG. 1, and the front logo is the pattern identification in the middle of the package in FIG. 1. Figure 10 For example, as shown in FIG. 1, 96 is used to indicate the cutting line, when the type is Tray Box, according to h×4 / 10:h×8 / 10 and (x1+x2) / 2: (y1+y2) / 2, the cutting line is h×4 / 10:h×8 / 10 and (x1+x2) / 2: (y1+y2) / 2. Figure 6

[0166] For example, as shown in FIG. 1, 96 is used to indicate the cutting line, when the type is Tray Box, according to h×4 / 10:h×8 / 10 and (x1+x2) / 2: (y1+y2) / 2, the cutting line is h×4 / 10:h×8 / 10 and (x1+x2) / 2: (y1+y2) / 2. Figure 11 ​​​​​​​​Cut out the identification area image, the identification area image is as shown in Figure 12 Cut out the seal feature area image according to h x 2 / 10: h x 6 / 10 and Cut out the seal feature area image according to h x 2 / 10: h x 6 / 10 and Figure 13

[0167] For another example, affine transformation is performed according to the dynamic cutting coordinates, and the normalized feature area image (the seal feature area image and the identification area image) is output.

[0168] The embodiments of the present disclosure can realize accurate image cutting, and extract the seal feature area and the identification area image. The affine transformation is used to realize rotation, scaling, translation and other operations of the image, which further improves the positioning and recognition accuracy. Through effective cutting and analysis, the key features of the target area can be quickly and accurately extracted, which provides guarantee for accurate detection of the packaging box in the subsequent process, and reduces the probability of missing detection of the packaging box.

[0169] In yet another example embodiment, before the seal feature area image and the identification area image are input into the detection model and the detection result is output, the packaging box detection method provided by the present disclosure can further include: determining the detection model based on the type of the packaging box to be detected.

[0170] In one embodiment, the detection model corresponding to the type of the packaging box to be detected is queried according to a preset relationship correspondence table. The relationship correspondence table stores the correspondence relationship between the type of the packaging box to be detected and the detection model.

[0171] In another embodiment, the detection model is determined based on the type of the packaging box to be detected, including: determining a weight file corresponding to the type of the packaging box to be detected; and obtaining the detection model according to a preset configuration file and the weight file.

[0172] In order to reduce the memory occupied by the detection model on the electronic device, the weight file can be determined according to the type of the packaging box to be detected, and the detection model can be obtained according to the preset configuration file and the weight file, so as to detect the packaging box by using the detection model.

[0173] In the embodiments of the present disclosure, the original image is preprocessed to obtain the seal feature area image and the identification area image, and then the seal feature area image and the identification area image are used for detection, so that the training sample amount and the computing power requirement of image recognition can be effectively reduced.

[0174] For example, the detection model is a lightweight deep learning model based on YOLO.

[0175] ​​​​It should be noted that a lightweight model refers to a model with fewer parameters and computational requirements compared to traditional deep learning models, thus consuming fewer resources and enabling efficient operation even on devices with lower hardware requirements. In other words, in this embodiment, the original image is first preprocessed, and the resulting seal feature region image and identification region image can be recognized using a lightweight model, saving computational power while ensuring recognition accuracy.

[0176] For example, a lightweight deep learning model based on YOLO uses the YOLOv3.6 algorithm for box detection.

[0177] For example, 200 photos (raw images) of Tray Box and 200 photos of Reel Box are taken using an industrial camera. The raw images are processed using methods S302 to S305 described above, resulting in 200 images of the labeled area and 200 images of the seal feature area for each type of packaging. 400 images of each Tray Box and Reel Box are manually labeled using labelimg. The model is then trained using the YOLOv3.6 algorithm, with 100 training iterations and a batch size of 16 images per iteration. Weight files for both Tray Box and Reel Box are then obtained. It should be noted that the weight files are part of the detection model; the detection model can be determined based on the weight files and configuration files.

[0178] For example, the detection model is determined based on the type of the packaging box to be detected, the seal feature area image and the identification area image are input into the detection model, and the detection result is output.

[0179] For example, based on YOLOv3.6, the yolov3-tiny.cfg configuration file, and combined with the weight file corresponding to the type of the packaging box to be detected obtained through training, recognition is performed. The confidence threshold (conf_thres) is 0.3, and the detection result is obtained, such as... Figure 14 and Figure 15 As shown, the confidence scores for the detection results of the identification area image and the seal feature area image are 0.98 and 0.99, respectively. Both confidence scores are greater than 0.3, indicating that the packaging box to be detected has passed the inspection.

[0180] In this embodiment, the detection model is determined according to the type of the packaging box to be detected, and dual-target detection is performed using the detection model. This helps to improve the detection accuracy of the packaging box to be detected, achieve high-precision detection of semi-transparent seals under low computing power resources, and solve the problem of missed detection of packaging boxes.

[0181] In yet another example embodiment, after the original image is acquired, the packaging box detection method provided by the present disclosure can further include: acquiring a packaging box identifier (Box ID). The packaging box identifier can be used to identify whether the packaging box to be detected belongs to the current site. The present disclosure does not make specific limitations on the form of the packaging box identifier. For example, the packaging box identifier can be a bar code or a two-dimensional code.

[0182] After the seal feature region image and the identification region image are input into the detection model and the detection result is output, the packaging box detection method provided by the present disclosure can further include: sending the packaging box identifier and the detection result to a manufacturing execution system (MES) to enable the MES to perform a verification according to the packaging box identifier and the detection result, and in the case that the detection result is passed and it is confirmed according to the packaging box identifier that the packaging box to be detected belongs to the current site, the verification result is passed.

[0183] For example, as shown in FIG. 16, Figure 16 The packaging box detection method provided by the present disclosure can include the following S1601 to S1604.

[0184] S1601, it is judged whether the detection result is passed. If the detection is passed, S1602 is executed; if the detection is not passed, the detection result is fed back, and the detection of the next packaging box to be detected is waited to be executed. For example, the double-target (identification region image and seal feature region image) detection result is “the confidence scores of the double-targets are 0.98 and 0.99 respectively”, both of the confidence scores are greater than 0.3, and the packaging box to be detected is detected to be passed. It should be noted that as long as one target is not detected to be passed, the detection is not passed.

[0185] S1602, the packaging box identifier is sent to the MES.

[0186] S1603, it is verified whether the packaging box identifier is in the current site. If the verification is passed, S1604 is executed; if the verification is not passed, the verification result is fed back, and the detection of the next packaging box to be detected is waited to be executed.

[0187] S1604, the MES passes the site and returns “verification success”.

[0188] For example, the Box ID and the detection result are returned to the MES through a Webservice protocol, the MES verifies whether the Box ID belongs to the current site, if the Box ID belongs to the current site and the detection result is passed, the MES passes the Box ID, and meanwhile, passes the information to the Packing Eye (a software name, installed on an electronic device) and the Packing Eye displays “verification success”; if it does not belong to the current site or the detection result is not passed, the Packing Eye will prompt an error message.

[0189] The embodiments of the present disclosure realize automatic return and real-time verification of the detection result of the packaging box by integrating the Webservice protocol, thereby improving the automation level and information flow efficiency of the production line. The system can automatically detect and return the Box ID and its detection result to the MES. After comparison and verification by the MES, it is determined whether to pass the packaging box to the next station. Through the real-time feedback mechanism, the Packing Eye timely displays "verification success" or error prompt, effectively avoiding the flow of unqualified products into the subsequent process, while ensuring production quality and improving efficiency.

[0190] The embodiments of the present disclosure return the Box ID and its detection result to the MES. After comparison and verification by the MES, it is determined whether to pass the packaging box to the next station. Through the real-time feedback mechanism, unqualified products are effectively prevented from flowing into the subsequent process, while ensuring production quality and improving efficiency.

[0191] It should be noted that the embodiments of the present disclosure can realize single packaging box detection time of not more than 1 second, and ensure that the system runs stably and continuously for 7x24 hours.

[0192] The present disclosure will be further described below through a specific embodiment.

[0193] In the exemplary embodiment, the packaging box detection process is as follows:

[0194] (1) Deploy the packaging box detection hardware.

[0195] Operation table: adopt a workbench with anti-static design (for example, the workbench is equipped with ESD (electrostatic discharge) grounding function), the size is LxWxH=120x80x160cm³, two positioning marks (such as green L-shaped corner marks) are pasted at a distance of 40 cm from the lower edge of the desktop of the workbench, 30 cm apart.

[0196] Imaging assembly: install an automatic focusing industrial camera (800 million pixels, maximum resolution 1920x1200, frame rate 30fps).

[0197] Illumination system: 15W (W) LED strip light source, located on the operation table, 30 cm from the desktop height.

[0198] (2) Reference positioning calibration.

[0199] Fix the camera on the support of the operation table, adjust the angle of the camera to ensure that the two kinds of packaging boxes and the positioning marks are completely located in the picture, and link the camera and the computer through USB.

[0200] (3) Model training.

[0201] It should be noted that the model training has been described in the foregoing, and will not be repeated here. The labeling tool is Labelimg, and the labeling types of the two types of packaging boxes are Tray Box: tray_box, seal and Reel Box: reel_box, seal. The model training platform can be NVIDIA GT730 (CUDA9.2), wherein NVIDIA GT730 is an NVIDIA graphics card for low-end graphics processing and some computing tasks, and CUDA9.2 is a parallel computing architecture provided by NVIDIA. The model training base network is Darknet-53 backbone, that is, Darknet-53 is used as the backbone network. The model training parameters: based on YOLOv3-tiny.cfg, modify epoch=100, batch-size=16 for training, and obtain the trained reel_box.pt and Tray_box.pt weight files, respectively.

[0202] (4) Packaging box detection and systematization.

[0203] Trigger: connect the code scanning gun and the computer through the USB, and trigger the photographing, image preprocessing and image detection by scanning the packaging box identifier (Box ID) by the code scanning gun.

[0204] Image preprocessing: preprocess the original picture according to S301 to S305 described above to obtain the type of the packaging box to be detected, the seal feature region image and the identification region image.

[0205] Image detection: according to the type of the packaging box to be detected, call the weight file obtained by training the corresponding image, use the YOLOv3.6 algorithm to detect the target region (seal feature region image and identification region image), and transmit the Box ID and the detection result to the MES.

[0206] Interaction with the MES system: after the MES obtains the Box ID and the detection result, the MES checks whether the Box ID belongs to the station, if it belongs to the station and the detection result is passed, the MES passes the Box ID through the station, and returns the passing information to the Packing Eye, and the Packing Eye displays “check success”; if it does not belong to the station or the detection result is not passed, the Packing Eye will prompt an error message.

[0207] Interface integration: develop the software interface based on PyQt5 and integrate the above steps, and complete the installation and deployment of the production line computer.

[0208] Based on the same inventive concept, this disclosure also provides a packaging box detection device, as described in the following embodiments. Since the principle by which this device solves the problem is similar to that of the method embodiments described above, the implementation of this device embodiment can refer to the implementation of the method embodiments described above, and repeated details will not be repeated.

[0209] Figure 17 This diagram illustrates a packaging box detection device according to an embodiment of the present disclosure, such as... Figure 17 As shown, the packaging box detection device includes: an acquisition module 171, a first determination module 172, a detection module 173, a second determination module 174, a cutting module 175, and an output module 176. The acquisition module 171 acquires an original image, which is obtained by photographing the packaging box to be detected. The original image includes a semi-transparent seal and a label. The first determination module 172 obtains an image registration reference based on the original image. The detection module 173 performs type detection on the original image based on the image registration reference to determine the type of the packaging box to be detected. The second determination module 174 performs blurring and edge feature extraction on the original image to obtain an image to be cut. The cutting module 175 cuts the image to be cut according to the image registration reference and the type of the packaging box to be detected, obtaining a seal feature area image and a label area image. The output module 176 inputs the seal feature area image and the label area image into the detection model and outputs the detection result.

[0210] In one embodiment, the original image includes a positioning marker for localization, the color of which is different from the color of the packaging box to be detected; the first determining module 172 can also be used to perform color space conversion on the original image to obtain a first image; generate a first mask based on a first HSV threshold range corresponding to the color of the positioning marker and the first image; extract color regions from the original image based on the first mask to obtain a second image; and determine the position coordinates of the positioning marker based on the second image.

[0211] In one embodiment, the second image includes a first target extraction region, which corresponds to a positioning marker. The first determining module 172 can also be used to convert the second image into a single-channel grayscale image and identify connected regions through a contour detection algorithm to obtain the contour of the first target extraction region; calculate the minimum bounding rectangle of the contour of the first target extraction region to obtain the position coordinates and physical size of the minimum bounding rectangle; determine the positioning point if the position coordinates and physical size of the minimum bounding rectangle meet the determination conditions; determine the position coordinates of the positioning marker based on the positioning point; wherein the determination conditions are that the physical size is greater than the size threshold and the position coordinates of the minimum bounding rectangle meet the position constraint conditions.

[0212] In an embodiment, the detection module 173 can be further configured to determine a second target extraction region on the original image according to the image registration reference; and determine the type of the to-be-detected packaging box according to the mark and the second target extraction region.

[0213] In an embodiment, the detection module 173 can be further configured to perform color space conversion on the second target extraction region to obtain a third image; generate a second mask according to a second HSV threshold range corresponding to the color of the mark and the third image; perform color region extraction on the second target extraction region according to the second mask to obtain a fourth image; calculate a pixel area of the mark according to the fourth image; calculate a ratio of the pixel area of the mark to a pixel area of the second target extraction region; and determine the type of the to-be-detected packaging box according to the ratio.

[0214] In an embodiment, the color of the mark is red, and the second HSV threshold range includes a first red threshold range and a second red threshold range; and the detection module 173 can be further configured to perform an OR operation on the first red threshold range and the second red threshold range to obtain a composite red threshold range; and generate the second mask according to the composite red threshold range and the third image.

[0215] In an embodiment, the detection module 173 can be further configured to determine that the type of the to-be-detected packaging box is a first type when the ratio is greater than or equal to an area ratio threshold value; and determine that the type of the to-be-detected packaging box is a second type when the ratio is less than the area ratio threshold value.

[0216] In an embodiment, the second determination module 174 can be further configured to perform Gaussian filtering on the original image to obtain a blurred image; and process the blurred image by using an edge detection algorithm to obtain the to-be-cut image.

[0217] In an embodiment, the cutting module 175 can be further configured to determine a cutting parameter according to the type of the to-be-detected packaging box and the image registration reference; and cut the to-be-cut image according to the cutting parameter to obtain a seal feature region image and a mark region image.

[0218] In an embodiment, before inputting the seal feature region image and the mark region image into the detection model and outputting a detection result, the output module 176 can be further configured to determine the detection model based on the type of the to-be-detected packaging box.

[0219] In an embodiment, the output module 176 can be further configured to determine a weight file corresponding to the type of the to-be-detected packaging box; and obtain the detection model according to a preset configuration file and the weight file.

[0220] In an embodiment, the detection model is a lightweight deep learning model based on YOLO.

[0221] In an embodiment, the number of positioning marks is 2, the shape of the positioning marks is L-shaped, and the packaging box to be detected is located between the two positioning marks.

[0222] In an embodiment, after obtaining the original image, the obtaining module 171 can also be configured to obtain a packaging box identifier; wherein, after inputting the seal feature region image and the identification region image into the detection model and outputting the detection result, the output module 176 can also be configured to send the packaging box identifier and the detection result to a manufacturing execution system (MES) to enable the MES to perform verification according to the packaging box identifier and the detection result, and in the case that the detection result is passed and the packaging box to be detected is confirmed to belong to the current site according to the packaging box identifier, the verification result is passed.

[0223] The packaging box detection device disclosed in the embodiments of the present disclosure can effectively reduce the training sample amount and the computing power requirement of image recognition. In addition, the localized region detection is realized through image cutting, which prevents missed detection. The present disclosure can realize high-precision detection of a semi-transparent seal under low computing power resources, and solve the problem of missed detection of packaging boxes.

[0224] According to still another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the steps of any of the above packaging box detection methods via execution of the executable instructions. Illustratively, the processor can be a graphics processing unit (GPU).

[0225] According to still another aspect of the present disclosure, a computer readable storage medium is provided, having a computer program stored thereon, the computer program being executed by a processor to implement the steps of any of the above packaging box detection methods.

[0226] According to still another aspect of the present disclosure, a computer program product is provided, comprising a computer program or computer instructions, which are loaded and executed by a processor to enable a computer to implement any of the above packaging box detection methods.

[0227] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), disk memory, CD-ROM, and optical memory; volatile memory can include random access memory (RAM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM). The term "computer-readable medium" includes, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and various other mediums capable of storing, containing, or carrying instruction(s) and / or data. Examples of a computer-readable medium include a floppy disk, a 3.5-inch flexible disk, a hard disk, a solid state drive, a magnetic tape, a cassette tape, an optical disk, a compact disk, a digital versatile disk, a Blu-ray disk, a memory card, a memory stick, and / or any other medium capable of storing software. The computer-readable medium can be a transitory or non-transitory computer-readable medium. The term "computer-readable medium" is also intended to include "wired" and / or "wireless" transmission media that transmit programming code. The term "computer-readable medium" is intended to also include various types of communications links and / or carriers, such as hardwired links, wireless links, communication links, and / or carrier waves that transmit programming code.

[0228] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. The disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such features that are evident from the specification and claims. The specification and examples are illustrative only and not restrictive of the disclosure. The true scope of the disclosure is set forth in the claims.

Claims

1. A package inspection method characterized by comprising: The method comprises the following steps: obtaining an original image, the original image being an image obtained by photographing a packaging box to be detected, the original image comprising a translucent seal and an identification, the original image comprising a positioning mark for positioning, the color of the positioning mark being different from the color of the packaging box to be detected; obtaining an image registration reference from the original image; performing type detection on the original image based on the image registration reference to obtain the type of the packaging box to be detected; performing blurring processing and edge feature extraction on the original image to obtain a to-be-cut image; cutting the to-be-cut image according to the image registration reference and the type of the packaging box to be detected to obtain a seal feature region image and an identification region image; inputting the seal feature region image and the identification region image into a detection model to output a detection result; wherein the step of obtaining an image registration reference from the original image comprises: performing color space conversion on the original image to obtain a first image; generating a first mask according to a first HSV threshold range corresponding to the color of the positioning mark and the first image; performing color region extraction on the original image according to the first mask to obtain a second image; and determining the position coordinates of the positioning mark according to the second image.

2. The package inspection method according to claim 1, characterized by, The second image comprises a first target extraction region corresponding to the positioning mark. The step of determining the position coordinates of the positioning mark according to the second image comprises: converting the second image into a single-channel grayscale image and identifying a connected region through a contour detection algorithm to obtain the contour of the first target extraction region; calculating the minimum circumscribed rectangle of the contour of the first target extraction region to obtain the position coordinates and physical size of the minimum circumscribed rectangle; determining a positioning point in the case where the position coordinates and physical size of the minimum circumscribed rectangle satisfy a determination condition; determining the position coordinates of the positioning mark according to the positioning point. The determination condition is that the physical size is greater than a size threshold value, and the position coordinates of the minimum circumscribed rectangle satisfy a position constraint condition.

3. The package inspection method according to claim 1, characterized by, The step of performing type detection on the original image based on the image registration reference to obtain the type of the packaging box to be detected comprises: determining a second target extraction region on the original image according to the image registration reference; determining the type of the packaging box to be detected according to the identification and the second target extraction region.

4. The package inspection method according to claim 3, wherein The step of determining the type of the packaging box to be detected according to the identification and the second target extraction region comprises: performing color space conversion on the second target extraction region to obtain a third image; generating a second mask according to a second HSV threshold range corresponding to the color of the identification and the third image; performing color region extraction on the second target extraction region according to the second mask to obtain a fourth image; calculating the pixel area of the identification according to the fourth image; calculating the ratio of the pixel area of the identification to the pixel area of the second target extraction region; determining the type of the packaging box to be detected according to the ratio.

5. The package inspection method according to claim 4, wherein The color of the mark is red, and the second HSV threshold range includes a first red threshold range and a second red threshold range; The second mask is generated according to the second HSV threshold range corresponding to the color of the mark and the third image, including: The first red threshold range and the second red threshold range are ORed to obtain a composite red threshold range; The second mask is generated according to the composite red threshold range and the third image.

6. The package inspection method according to claim 4, wherein The type of the packaging box to be detected is determined according to the ratio, including: In the case that the ratio is greater than or equal to an area ratio threshold, the type of the packaging box to be detected is determined as a first type; In the case that the ratio is less than the area ratio threshold, the type of the packaging box to be detected is determined as a second type.

7. The package inspection method according to claim 1, wherein The original image is blurred and the edge features are extracted to obtain a to-be-cut image, including: The original image is subjected to Gaussian filtering to obtain a blurred image; The blurred image is processed using an edge detection algorithm to obtain the to-be-cut image.

8. The package inspection method according to claim 1, wherein According to the image registration reference and the type of the packaging box to be detected, the to-be-cut image is cut to obtain a seal feature region image and a mark region image, including: According to the type of the packaging box to be detected and the image registration reference, a cutting parameter is determined; The to-be-cut image is cut according to the cutting parameter to obtain the seal feature region image and the mark region image.

9. The package inspection method according to claim 1, wherein Before the seal feature region image and the mark region image are input into a detection model to output a detection result, the method further includes: A detection model is determined based on the type of the packaging box to be detected.

10. The package inspection method according to claim 9, wherein The detection model is determined based on the type of the packaging box to be detected, including: A weight file corresponding to the type of the packaging box to be detected is determined; The detection model is obtained according to a preset configuration file and the weight file.

11. The package inspection method according to claim 2 or 3, characterized by, The number of positioning marks is 2, the shape of the positioning mark is L-shaped, and the packaging box to be detected is located between the two positioning marks.

12. The package inspection method according to claim 10, wherein After the original image is obtained, the method further includes: A packaging box identifier is obtained; After the seal feature region image and the mark region image are input into a detection model to output a detection result, the method further includes: The packaging box identifier and the detection result are sent to a manufacturing execution system (MES) to enable the MES to perform verification according to the packaging box identifier and the detection result, and the verification result is passed in the case that the detection result is passed and the packaging box to be detected is confirmed to belong to the current site according to the packaging box identifier.

13. A carton detection apparatus, characterised in that, Including: An acquisition module is configured to acquire an original image, the original image being an image obtained by photographing a packaging box to be detected, the original image including a translucent seal and a mark, and the original image including a positioning mark for positioning, the color of the positioning mark being different from the color of the packaging box to be detected; A first determination module is configured to obtain an image registration reference according to the original image; The detection module is configured to perform type detection on the original image based on the image registration reference, to obtain a type of the packaging box to be detected. The second determination module is configured to perform blurring processing and edge feature extraction on the original image, to obtain a to-be-cut image. The cutting module is configured to cut the to-be-cut image according to the image registration reference and the type of the packaging box to be detected, to obtain a seal feature region image and an identification region image. The output module is configured to input the seal feature region image and the identification region image into a detection model, and output a detection result. The first determination module is further configured to perform color space conversion on the original image, to obtain a first image; generate a first mask according to a first HSV threshold range corresponding to a color of the positioning mark and the first image; and perform color region extraction on the original image according to the first mask, to obtain a second image. The position coordinates of the positioning mark are determined according to the second image.

Images