Food bag sealing visual inspection method and device

Abstract

This invention provides a visual inspection method and apparatus for food sealing bags, relating to the field of visual inspection technology. The method includes: when a mark detector detects a mark on the food sealing bag, triggering a visual sensor to acquire image information of the food sealing bag according to a preset image acquisition delay; based on multi-channel data in the image information, extracting the target contour information corresponding to the food to be packaged and the mark detection area corresponding to the mark; generating a food detection area based on the mark detection area; and controlling a sealing bag cutting mechanism to perform a cutting action on the food sealing bag according to the relative positional relationship between the target contour information and the mark detection area and the food detection area. This invention is adaptively compatible with multiple packaging materials, robustly segments multi-colored food targets, and achieves precise anti-cutting decisions.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a visual inspection method and apparatus for food sealed bags. Background Technology

[0002] In the field of automated food packaging, food items need to be individually placed into individual packaging bags made of continuous roll film, and then heat-sealed and cut at designated locations. This process typically relies on manual placement of the food into the packaging film on a conveyor belt, followed by sealing and cutting by a packaging machine. To ensure accurate cutting, the outer surface of the packaging film is periodically printed with markings for photoelectric positioning (hereinafter referred to as "black marks") along the film's movement direction, and the cutting action is strictly performed according to the positions of the black marks.

[0003] However, the following technical bottlenecks exist in actual production: First, the black label has poor imaging compatibility: When the packaging film uses opaque materials such as aluminum foil composite film, the black label is printed on the back of the film and cannot be imaged by conventional backlight transmission. Existing technologies mostly use a single lighting mode (such as only backlight or only front lighting), which makes it difficult to take into account both transparent film and aluminum film materials. This causes the vision system to be unable to reliably identify the black label under aluminum film conditions, which in turn leads to miscutting.

[0004] Second, the robustness of food target segmentation is insufficient: the colors of food to be packaged are diverse (such as yellow, cyan, brown, black, etc.), and their optical characteristics vary significantly with the category, maturity and processing technology; when the color of the food is similar to the gray level of the packaging film background (for example, brown food placed in the light gray reflective area of ​​aluminum film), traditional image segmentation methods are difficult to reliably separate the food area from the background, resulting in missed detection or misjudgment, affecting the reliability of anti-cutting decisions.

[0005] Third, the detection results are disconnected from the mechanical execution: Although the existing solution can output food location information, it lacks a quantitative judgment mechanism for the core risk of whether the food object physically covers the cutting path. It only relies on rough location judgment and cannot effectively prevent food from being cut, the seal from failing, or adjacent packaging from being contaminated. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a visual inspection method and device for food sealed bags, which can adaptively and compatiblely handle multiple types of packaging materials, robustly segment multi-colored food targets, and achieve accurate anti-cutting decisions.

[0007] In a first aspect, the present invention provides a visual inspection method for food sealing bags, applied to food sealing equipment. The food sealing equipment includes a conveyor belt, a visual inspection mechanism, and a sealing bag cutting mechanism. Continuous food sealing bags are placed on the conveyor belt. Marks for marking cutting positions are periodically printed on the outer surface of the food sealing bags along the film-moving direction. The film segment between two adjacent marks constitutes an independent packaging unit. The food to be packaged is placed on the inner surface of the packaging unit. The visual inspection mechanism includes a mark detector, a mark reflection structure, and a visual sensor, comprising: When the mark detector detects a mark on the food sealing bag, the visual sensor is triggered to collect image information of the food sealing bag according to the preset image capture delay. The content displayed in the image information includes at least the mark located on the outer surface of the food sealing bag and the food to be packaged located on the inner surface of the food sealing bag. The presentation of the mark in the image information depends on the optical effect of the mark's reflective structure. Based on multi-channel data in the image information, the target contour information corresponding to the food to be packaged and the sign detection area corresponding to the sign are extracted respectively. Generate food detection areas based on marker detection areas; Based on the relative positional relationship between the target contour information and the mark detection area and the food detection area, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag.

[0008] In one embodiment, the marker reflective structure includes a prism, at least two symmetrically arranged light sources, and backlights partitioned along the film-moving direction; wherein, Prisms are used to allow the optical information of a mark to enter the field of view of a vision sensor via a surface reflection path when the food sealing bag is made of an opaque material. At least two light sources project light onto the food seal bag in a non-normal incidence manner; Backlighting is used to illuminate only the area near the logo when the food sealing bag is made of opaque material.

[0009] In one implementation, based on multi-channel data in the image information, the marker detection region corresponding to the marker is extracted, including: Obtain the region of interest (ROI) from the image information; Based on the region of interest marked, connected component analysis is performed sequentially on the multi-channel data in the image information; Determine whether the connected component analysis has extracted candidate regions that meet the preset area constraints; If so, then generate the corresponding sign detection region based on the center coordinates of the candidate region; If not, then based on the region of interest of the sign, subpixel edge detection is performed sequentially on the multi-channel data in the image information to obtain the subpixel coordinates of the two sides of the sign, and the sign detection region corresponding to the sign is generated according to the subpixel coordinates.

[0010] In one implementation, based on the marked region of interest, connected component analysis is sequentially performed on multi-channel data in the image information, and it is determined whether the connected component analysis has extracted candidate regions that satisfy a preset area constraint, including: Extract the current channel data corresponding to the region of interest marked in the image information; Perform connected component analysis on the grayscale image corresponding to the current channel data to obtain candidate regions; If the area of ​​the candidate region is less than the first area threshold, it is determined that the preset area constraint is not met, and the next channel data corresponding to the region of interest is extracted from the image information. If the area of ​​the candidate region is greater than the first area threshold, it is determined that the preset area constraint is met.

[0011] In one implementation, generating a marker detection region corresponding to a marker based on the center coordinates of the candidate region includes: Determine the center coordinates and width of the minimum bounding rectangle of the candidate region; Using the center coordinates of the minimum bounding rectangle as a reference point, a rectangular region is generated by extending along the film-running direction. The width of the rectangular region is equal to the width of the minimum bounding rectangle, and the height of the rectangular region is a specified multiple of the width of the minimum bounding rectangle. The rectangular area is used as the marker detection area. The center coordinates of the marker detection area overlap with the center coordinates of the smallest bounding rectangle, and the long side of the marker detection area is parallel to the film walking direction.

[0012] In one implementation, based on multi-channel data in the image information, target contour information corresponding to the food to be packaged is extracted, including: The image information is semantically segmented to obtain the first contour information and color information corresponding to the food to be packaged; Based on the pre-configured mapping relationship between color information and channel data, the target channel data is determined from the multi-channel data in the image information, and the target channel data is analyzed based on connected components to obtain the second contour information corresponding to the food to be packaged. The union of the first contour information and the second contour information is taken as the target contour information corresponding to the food to be packaged.

[0013] In one implementation, generating a food detection area based on a marker detection area includes: Using the geometric center of the mark detection area as the reference origin, offset by a specified distance along the preset food placement direction to obtain the area's starting baseline; Based on the regional starting baseline, a rectangular food detection area is generated. The extension direction of the food detection area is parallel to the film feeding direction and covers the landing point range of the food to be packaged under normal delivery conditions.

[0014] In one embodiment, based on the relative positional relationship between the target contour information and the mark detection area and the food detection area, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag, including: Based on the relative positional relationship between the target contour information and the food detection area, determine whether the target contour information is located within the food detection area; If not, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag; If so, based on the relative positional relationship between the target contour information and the mark detection area, determine whether to control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag.

[0015] In one implementation, determining whether to control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag based on the relative positional relationship between the target contour information and the mark detection area includes: Determine the overlapping pixel area between the target contour information and the marker detection area; Determine whether the area of ​​overlapping pixels is greater than the second area threshold; If so, then determine that the food to be packaged meets the cutting conditions, and prohibit the control of the sealing bag cutting mechanism from performing cutting actions on the food sealing bag; If not, the control of the sealing bag cutting mechanism will perform a cutting action on the food sealing bag.

[0016] Secondly, the present invention also provides a visual inspection device for food sealing bags, applied to food sealing equipment. The food sealing equipment includes a conveyor belt, a visual inspection mechanism, and a sealing bag cutting mechanism. A continuous series of food sealing bags are placed on the conveyor belt. Marks for marking cutting positions are periodically printed on the outer surface of the food sealing bags along the film-moving direction. The film segment between two adjacent marks constitutes an independent packaging unit. The food to be packaged is placed on the inner surface of the packaging unit. The visual inspection mechanism includes a mark detector, a mark reflection structure, and a visual sensor, comprising: The image acquisition module is used to trigger the visual sensor to acquire image information of the food sealing bag according to a preset image acquisition delay when the mark detector detects the mark on the food sealing bag. The content displayed in the image information includes at least the mark located on the outer surface of the food sealing bag and the food to be packaged located on the inner surface of the food sealing bag. The presentation of the mark in the image information depends on the optical effect of the mark's reflective structure. The first region extraction module is used to extract the target contour information and the sign detection area corresponding to the sign of the food to be packaged based on the multi-channel data in the image information. The second region generation module is used to generate food detection regions based on the mark detection regions. The control module is used to control the sealing bag cutting mechanism to perform cutting actions on the food sealing bag based on the relative positional relationship between the target contour information and the mark detection area and the food detection area.

[0017] This invention provides a visual inspection method and apparatus for food sealing bags, applied to food sealing equipment. The food sealing equipment includes a conveyor belt, a visual inspection mechanism, and a sealing bag cutting mechanism. Continuous food sealing bags are placed on the conveyor belt. Marks for marking cutting positions are periodically printed on the outer surface of each food sealing bag along the film-moving direction. The film segment between two adjacent marks constitutes an independent packaging unit. The food to be packaged is placed on the inner surface of the packaging unit. The visual inspection mechanism includes a mark detector, a mark reflection structure, and a visual sensor. When the mark detector detects a mark on the food sealing bag, the visual sensor is triggered to acquire image information of the food sealing bag according to a preset image acquisition delay. The image information displays at least the mark located on the outer surface of the food sealing bag and the food to be packaged located on the inner surface of the food sealing bag. The representation of the mark in the image information depends on the optical effect of the mark reflection structure. Then, based on the multi-channel data in the image information, the target contour information corresponding to the food to be packaged and the mark detection area corresponding to the mark are extracted respectively. Next, a food detection area is generated based on the mark detection area. Finally, according to the relative positional relationship between the target contour information and the mark detection area and the food detection area, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag. The above method effectively introduces the optical information of the marking through the marking reflection structure under the condition of aluminum foil composite film, so that the marking on the outer surface and the food to be packaged on the inner surface can be presented simultaneously in a single frame image, taking into account both transparent and opaque packaging materials; based on multi-channel data, the target contour information of the marking detection area and the food to be packaged is extracted separately, improving the recognition robustness of complex colored foods under different backgrounds; relying on the marking detection area, the food detection area is dynamically generated, and based on the relative positional relationship between the target contour information and the marking detection area and the food detection area, the cutting action is precisely controlled, effectively avoiding food damage, sealing failure and contamination of adjacent packaging, improving the accuracy of visual inspection and the reliability of packaging operations.

[0018] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a visual inspection method for food sealing bags provided in an embodiment of the present invention; Figure 2 A schematic diagram of a marker reflection structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating a scenario where target contour information does not fall within the food detection area, as provided in an embodiment of the present invention. Figure 4 A schematic diagram illustrating a material cutting process according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating a non-cutting scenario provided by an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a visual inspection device for food sealing bags provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Currently, the following technical bottlenecks exist in actual production: poor compatibility of black mark imaging, insufficient robustness of food target segmentation, and disconnect between detection results and mechanical execution. Based on this, the present invention provides a visual inspection method and device for food sealed bags, which can adaptively and compatiblely handle multiple types of packaging materials, robustly segment multi-colored food targets, and achieve accurate anti-cutting decisions.

[0024] To facilitate understanding of this embodiment, a visual inspection method for food sealing bags disclosed in this invention will first be described in detail. This method is applied to food sealing equipment, which includes a conveyor belt, a visual inspection mechanism, and a sealing bag cutting mechanism. Continuous food sealing bags are placed on the conveyor belt. Marks for marking cutting positions are periodically printed on the outer surface of the food sealing bags along the film-moving direction. The film segment between two adjacent marks constitutes an independent packaging unit. The food to be packaged is placed on the inner surface of the packaging unit. The visual inspection mechanism includes a mark detector, a mark reflection structure, and a visual sensor. (See also...) Figure 1 The diagram shows a flowchart of a visual inspection method for sealed food bags. The method mainly includes the following steps S102 to S108: Step S102: When the mark detector detects the mark on the food sealed bag, the visual sensor is triggered to collect image information of the food sealed bag according to the preset image capture delay.

[0025] The image information displayed includes at least a mark on the outer surface of the food sealing bag and the food to be packaged on the inner surface of the food sealing bag. The presentation of the mark in the image information depends on the optical effect of the mark reflection structure. The food to be packaged may include granular, blocky, or irregularly shaped solid foods such as plums, preserved plums, green plums, candied fruit, dried fruit, nuts, dried fruit, candy, chocolate chips, dried meat, dried fish, and seaweed sheets. The mark is also known as a black label. In one embodiment, a mark detector detects the mark on the outer surface of the food sealing bag and sends a trigger signal to a vision sensor when the mark is detected. This triggers the vision sensor to acquire image information of the food sealing bag according to a preset image acquisition delay. During the image acquisition process, the mark reflection structure guides the optical information of the mark printed on its outer surface to the field of view of the vision sensor via a surface reflection path when the food sealing bag is made of opaque material. This allows the vision sensor to acquire a usable image containing the mark in a single frame.

[0026] Step S104: Based on the multi-channel data in the image information, extract the target contour information corresponding to the food to be packaged and the sign detection area corresponding to the sign.

[0027] The multi-channel color data can be R-channel data, G-channel data, B-channel data, or HSV-channel data, etc. In one implementation, semantic segmentation and connected component analysis based on multi-channel data can be combined to extract the fine contour of the target outline corresponding to the food to be packaged, and connected component analysis and sub-pixel edge detection based on multi-channel data can be combined to extract the mark detection region corresponding to the mark.

[0028] Step S106: Generate a food detection area based on the marker detection area.

[0029] The food detection area is used to define the expected range of the food to be packaged in the image, serving as a spatial reference for determining whether the food to be packaged is correctly placed within the packaging unit; this area is generated based on the mark detection area to ensure that its position has a definite geometric relationship with the mark.

[0030] Step S108: Based on the relative positional relationship between the target contour information and the mark detection area and the food detection area, control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag.

[0031] In one implementation, the relative positional relationship between the target contour information and the food detection area can be used to determine whether the food to be packaged is located within the food detection area. If there is no food to be packaged within the food detection area, the cutting action can be performed directly. If there is food to be packaged within the food detection area, it is necessary to further determine whether there is a cutting situation based on the relative positional relationship between the target contour information and the mark detection area. If there is, the cutting action is prohibited and the process returns to step S102. If there is no cutting, the cutting action can be performed.

[0032] The visual inspection method for food sealing bags provided in this invention effectively introduces the optical information of the marking through a marking reflection structure in the case of aluminum foil composite film. This allows a single frame image to simultaneously display the marking on the outer surface and the food to be packaged on the inner surface, accommodating both transparent and opaque packaging materials. Based on multi-channel data, the method extracts the target contour information of the marking detection area and the food to be packaged, improving the robustness of recognition of complex-colored foods under different backgrounds. The method dynamically generates a food detection area based on the marking detection area and, according to the relative positional relationship between the target contour information and the marking and food detection areas, achieves precise control of the cutting action, effectively preventing food damage, sealing failure, and contamination of adjacent packaging, thus improving the accuracy of visual inspection and the reliability of packaging operations.

[0033] For ease of understanding, this invention provides an implementation method for visual inspection of food sealed bags, as detailed below: (i) When the mark detector detects the mark on the food sealed bag, the visual sensor is triggered to collect the image information of the food sealed bag according to the preset photo capture delay.

[0034] Specifically, packaging personnel place the food to be packaged sequentially onto the inner surface of a continuous food sealing bag on the conveyor belt, ensuring that the food falls into the packaging unit between two adjacent markers. Color mark sensors (i.e., marker detectors) are installed at designated locations on the conveyor belt to detect markers in real time. When a marker passes the color mark sensor, it outputs a trigger signal to a vision sensor. Due to the physical distance between the color mark sensor and the vision sensor, a preset image capture delay needs to be set to ensure that the captured image includes the marker and the corresponding food to be packaged.

[0035] Furthermore, when the food sealing bag is an aluminum foil composite film, a marking reflective structure is activated, such as... Figure 2 The diagram illustrates a sign reflection structure, which includes a prism, at least two symmetrically arranged light sources, and backlights partitioned along the film-laying direction. The prism allows the optical information of the sign to enter the field of view of the vision sensor via a surface reflection path when the food sealing bag is made of an opaque material. The at least two light sources project light onto the food sealing bag in a non-normal incidence manner. The backlights, when the food sealing bag is made of an opaque material, activate only the partitions near the sign to provide localized illumination. The opaque material includes at least aluminum film. For example, during the operation of food packaging equipment, because the aluminum foil composite film is opaque to visible light, and the black mark is printed on the back of the film, it cannot form a recognizable feature in the image through front transmission or direct illumination. Therefore, a prism is used to guide the surface reflected light of the black mark area to the field of view of the visual sensor. Since the surface of the aluminum foil composite film has high specular reflectivity, if vertical incident illumination is used, tiny wrinkles on the film surface will cause drastic fluctuations in local reflected light intensity, forming an interference area with alternating light and dark areas. When the gray value of this interference area is close to the gray value characteristics of the food to be packaged, it is easily misidentified as a food target, causing misjudgment. Therefore, in this embodiment of the invention, two strip light sources illuminate the film surface at a non-perpendicular angle to alleviate the uneven reflection caused by wrinkles and improve the above-mentioned problems. The zone closest to the black mark position in the three-zone backlight is activated to provide local illumination to the area where the black mark is located, enhancing its contrast and detectability in the image. The three work together to ensure stable acquisition of a detectable black mark image in a single frame, providing a reliable benchmark for subsequent positioning and cutting control.

[0036] (ii) Extracting the sign detection region corresponding to the sign based on multi-channel data in the image information. This includes the following steps: (1) Obtain the region of interest (ROI) of the sign in the image information. In one example, the ROI is detected based on a manually pre-calibrated sign, and the sign is located in the image within this region. The ROI is set based on the printing position of the sign on the packaging film to limit the sign search range and improve the positioning efficiency and anti-interference ability.

[0037] (2) Based on the region of interest, perform connected component analysis on the multi-channel data in the image information in sequence, and determine whether the connected component analysis has extracted candidate regions that meet the preset area constraints.

[0038] In one implementation, the current channel data corresponding to the region of interest is extracted from the image information; connected component analysis is performed on the grayscale image corresponding to the current channel data to obtain candidate regions; if the area of ​​the candidate region is less than a first area threshold, it is determined that the preset area constraint is not met, and the next channel data corresponding to the region of interest is extracted from the image information; if the area of ​​the candidate region is greater than the first area threshold, it is determined that the preset area constraint is met.

[0039] Specifically, given that the input image information is a three-channel color image, in order to improve the detection robustness of the mark under different lighting and film material conditions, connected component analysis is performed on the grayscale images of the R, G, and B color channels in sequence. If a candidate region that meets the preset area constraint is identified in the R channel, the subsequent channel processing is terminated; otherwise, the same analysis process is repeated in the G and B channels in sequence until a candidate region that meets the preset area constraint is successfully extracted in any channel.

[0040] (3) If so, generate the corresponding sign detection region based on the center coordinates of the candidate region. Obtain the candidate region and calculate the center coordinates (RowCenter, ColCenter) and width of its minimum bounding rectangle; using the center coordinates of the minimum bounding rectangle as the reference point, extend along the film-moving direction to generate a rectangular region. The width of the rectangular region is equal to the width of the minimum bounding rectangle, and the height of the rectangular region is a specified multiple of the width of the minimum bounding rectangle (such as 0.3 to 0.6 times); this rectangular region is the sign detection region. The center coordinates of the detected sign region overlap with the center coordinates of the minimum bounding rectangle, and the long side of the detected sign region is parallel to the film-moving direction.

[0041] (4) If not, then based on the region of interest of the sign, perform subpixel edge detection sequentially on the multi-channel data in the image information to obtain the subpixel coordinates of the two sides of the sign, and generate the sign detection region corresponding to the sign based on the subpixel coordinates. If no candidate region satisfying the area constraint is extracted in the R, G, and B channels, then perform subpixel edge detection (i.e., caliper method) sequentially on the grayscale image of the R, G, and B color channels in the region of interest of the sign to locate the two sides of the sign, and determine the position of the black mark based on the geometric center of the two sides of the edge; if no valid edge pair is obtained in the three-channel edge detection, then the image acquisition is deemed invalid, and an alarm for abnormal membrane position shift or shooting delay is triggered; after successfully locating the sign, the sign detection region for cutting control is generated based on the sign positioning result.

[0042] (III) Extract the target contour information corresponding to the food to be packaged based on multi-channel data in the image information. This includes the following steps: (1) Perform semantic segmentation on the image information to obtain the first contour information and color information corresponding to the food to be packaged.

[0043] Optionally, the image information can be semantically segmented using the Halcon semantic segmentation algorithm to obtain the first contour information and color information corresponding to the food to be packaged. The specific implementation process is as follows: Using a pre-trained semantic segmentation model in Halcon (such as U-Net, DeepLab, etc.), the input image of the food to be packaged is input into the model, the inference process is executed, and a semantic segmentation result map is generated. This result map is a matrix of the same size as the input image, where the value of each element represents the category to which the pixel belongs (such as food, background, etc.); Extracting the first contour information: The semantic segmentation result map is binarized, and pixels belonging to the food category are set to 1, and other pixels are set to 0, resulting in a binarized image. The edge detection algorithm (such as the Canny operator) or morphological operations (such as the contour extraction operator) in Halcon is used to process the binarized image to extract the first contour information of the food to be packaged; Extracting color information: Based on the semantic segmentation result map, the main regions of the food to be packaged are determined, and color analysis functions in Halcon (such as calculating the average RGB value, HSV value, or color histogram of pixels, etc.) are used in these regions to extract the color information of the food to be packaged.

[0044] (2) Based on the mapping relationship between pre-configured color information and channel data, the target channel data is determined from the multi-channel data in the image information, and the target channel data is analyzed based on connected components to obtain the second contour information corresponding to the food to be packaged.

[0045] A mapping table between color information and color channels is pre-established. Specifically: when the color information of the food to be packaged shows that the mean value of the R channel is significantly higher than that of the G and B channels, and the variance of the R channel is within a preset range, it is mapped to the R channel; when the mean value of the G channel is significantly higher than that of the R and B channels, it is mapped to the G channel; and when the mean value of the B channel is significantly higher than that of the R and G channels, it is mapped to the B channel. This mapping relationship is stored in the system configuration parameters. During runtime, the color information obtained from the aforementioned semantic segmentation is acquired, and the target channel data is determined by looking up the table. The grayscale image of this channel is extracted, and Otsu adaptive threshold binarization is performed. Subsequently, connected component analysis (e.g., blob analysis) is executed to filter connected regions that satisfy area constraints, which are then used as the second contour information of the food to be packaged.

[0046] (3) Take the union of the first contour information and the second contour information as the target contour information corresponding to the food to be packaged.

[0047] (iv) Generating food testing areas based on marker detection areas. This includes the following steps: Using the geometric center of the marking detection area as the reference origin, offset by a specified distance along the preset food placement direction to obtain the area's starting baseline; based on the area's starting baseline, a rectangular food detection area is generated, with the extension direction of the food detection area parallel to the film-moving direction and covering the landing point range of the food to be packaged under normal placement conditions.

[0048] Using the geometric center coordinates (Row0, Col0) of the mark detection area as the reference origin, offset by a specified pixel distance ΔRow along the preset food placement direction, i.e., the forward or reverse direction of the film feeding direction (determined by the actual feeding position on the production line, for example, the downstream side of the black mark is the forward direction), to obtain the row coordinates of the starting baseline Row1 = Row0 + ΔRow; this baseline is a horizontal straight line (parallel to the image column direction) that runs through the image width; with Row1 as the upper boundary (or lower boundary, set according to the offset direction), a rectangular area with a height of H and a width of W is generated, where H ranges from 15 to 60 pixels, and W covers the entire horizontal span of the packaging unit; this rectangular area is the food detection area, and its long side is consistent with the image row direction, so its extension direction is parallel to the film feeding direction, used to cover the typical landing point distribution of the food to be packaged in the film feeding direction.

[0049] (v) Based on the relative positional relationship between the target contour information and the mark detection area and the food detection area, control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag. Specifically, this includes the following steps: (1) Determine whether the target contour information is located within the food detection area based on the relative positional relationship between the target contour information and the food detection area.

[0050] Calculate the minimum bounding rectangle (Row1, Col1, Row2, Col2) corresponding to the target contour information; determine whether the geometric center point ((Row1+Row2) / 2, (Col1+Col2) / 2) of this rectangle falls within the boundary of the food inspection area; if the center point coordinates satisfy: Row1≤ (Row1+Row2) / 2 ≤ Row2 (Row direction range of food inspection area); And Col1≤ (Col1+Col2) / 2≤ Col2 (the range of the food testing area column direction); If the target outline information is located within the food detection area, it is determined that the target is not within the detection area; otherwise, it is determined that the target is not within the detection area.

[0051] Furthermore, if the number of pixels intersecting the overall pixel set of the target contour information and the food detection area exceeds a preset threshold, it is also considered to be located within that area. In practice, either of the two determination methods can be used or used in combination.

[0052] (2) If not, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag. For example, see [link to example]. Figure 3 The diagram shows a scenario where the target outline information does not fall within the food detection area, meaning the food detection area is empty and there is no cutting action, so the cutting action can be performed directly.

[0053] (3) If so, determine whether to control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag based on the relative positional relationship between the target contour information and the mark detection area. Specifically, determine the overlapping pixel area between the target contour information and the mark detection area; determine whether the overlapping pixel area is greater than the second area threshold; if so, determine that the food to be packaged meets the cutting conditions, and prohibit the sealing bag cutting mechanism from performing a cutting action on the food sealing bag; if not, control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag.

[0054] A pixel-by-pixel logical AND operation is performed between the binary mask image corresponding to the target contour information (pixel value of 1 represents the food area, 0 represents the background) and the binary mask of the sign detection area (pixel value of 1 represents the rectangular area, 0 represents the outside), resulting in an intersection image. The total number of pixels with a value of 1 in the intersection image is counted as the overlapping pixel area. This area is compared with a second area threshold: the second area threshold is a fixed pixel value based on the black mark width, ranging from 5% to 20% of the number of pixels corresponding to the black mark width; if the overlapping pixel area is greater than this threshold (e.g., ...), the overlapping pixel area is considered to be larger than the threshold. Figure 4 (As shown in the schematic diagram of one cutting situation), if the food to be packaged covers the black label area, it is determined that there is a risk of cutting, an anti-cut signal is generated and output to the sealing bag cutting mechanism, prohibiting it from performing this cutting action; otherwise (such as... Figure 5 The diagram shows a non-cutting situation. No anti-cutting signal is output, and the sealing bag cutting mechanism performs cutting according to the predetermined rhythm.

[0055] In summary, the embodiments of the present invention have at least the following characteristics: 1) Through the collaborative optical design of the sign reflection structure, symmetrical strip light source and zoned controllable backlight, adaptive compatibility with various packaging materials such as transparent film and aluminum foil composite film is achieved, effectively solving common problems in industrial sites such as the black mark not being visible under opaque film and unstable imaging on reflective film surface. 2) A hybrid detection strategy that integrates deep learning semantic segmentation and traditional connected component analysis balances the robustness of target recognition with real-time computation, significantly improving the segmentation accuracy and localization reliability of multi-colored and multi-shaped foods in complex backgrounds.

[0056] Based on the foregoing embodiments, this invention provides a visual inspection device for food sealing bags, applied to food sealing equipment. The food sealing equipment includes a conveyor belt, a visual inspection mechanism, and a sealing bag cutting mechanism. Continuous food sealing bags are placed on the conveyor belt. Marks for marking cutting positions are periodically printed on the outer surface of the food sealing bags along the film-moving direction. The film segment between two adjacent marks constitutes an independent packaging unit. The food to be packaged is placed on the inner surface of the packaging unit. The visual inspection mechanism includes a mark detector, a mark reflection structure, and a visual sensor. (See also...) Figure 6 The diagram shows a structural schematic of a visual inspection device for sealed food bags. The device mainly includes the following parts: The image acquisition module 602 is used to trigger the visual sensor to acquire image information of the food sealing bag according to a preset image acquisition delay when the mark detector detects the mark on the food sealing bag. The content displayed in the image information includes at least the mark located on the outer surface of the food sealing bag and the food to be packaged located on the inner surface of the food sealing bag. The presentation of the mark in the image information depends on the optical effect of the mark reflection structure. The first region extraction module 604 is used to extract the target contour information and the sign detection area corresponding to the sign of the food to be packaged based on the multi-channel data in the image information. The second region generation module 606 is used to generate a food detection region based on the mark detection region; The control module 608 is used to control the sealing bag cutting mechanism to perform cutting actions on the food sealing bag based on the relative positional relationship between the target contour information and the mark detection area and the food detection area.

[0057] The visual inspection device for food sealing bags provided in this invention effectively introduces optical information of the markings into the aluminum foil composite film condition through a marking reflection structure. This allows the markings on the outer surface and the food to be packaged on the inner surface to be simultaneously displayed in a single frame image, accommodating both transparent and opaque packaging materials. Based on multi-channel data, the device extracts the target contour information of the marking detection area and the food to be packaged, improving the robustness of recognition of complex colored foods under different backgrounds. It dynamically generates a food detection area based on the marking detection area and, according to the relative positional relationship between the target contour information and the marking and food detection areas, achieves precise control of the cutting action, effectively preventing food damage, sealing failure, and contamination of adjacent packaging, thus improving the accuracy of visual inspection and the reliability of packaging operations.

[0058] In one embodiment, the marker reflective structure includes a prism, at least two symmetrically arranged light sources, and backlights partitioned along the film-moving direction; wherein, Prisms are used to allow the optical information of a mark to enter the field of view of a vision sensor via a surface reflection path when the food sealing bag is made of an opaque material. At least two light sources project light onto the food seal bag in a non-normal incidence manner; Backlighting is used to illuminate only the area near the logo when the food sealing bag is made of opaque material.

[0059] In one embodiment, the first region extraction module 604 is specifically used for: Obtain the region of interest (ROI) from the image information; Based on the region of interest marked, connected component analysis is performed sequentially on the multi-channel data in the image information; Determine whether the connected component analysis has extracted candidate regions that meet the preset area constraints; If so, then generate the corresponding sign detection region based on the center coordinates of the candidate region; If not, then based on the region of interest of the sign, subpixel edge detection is performed sequentially on the multi-channel data in the image information to obtain the subpixel coordinates of the two sides of the sign, and the sign detection region corresponding to the sign is generated according to the subpixel coordinates.

[0060] In one embodiment, the first region extraction module 604 is specifically used for: Extract the current channel data corresponding to the region of interest marked in the image information; Perform connected component analysis on the grayscale image corresponding to the current channel data to obtain candidate regions; If the area of ​​the candidate region is less than the first area threshold, it is determined that the preset area constraint is not met, and the next channel data corresponding to the region of interest is extracted from the image information. If the area of ​​the candidate region is greater than the first area threshold, it is determined that the preset area constraint is met.

[0061] In one embodiment, the first region extraction module 604 is specifically used for: Determine the center coordinates and width of the minimum bounding rectangle of the candidate region; Using the center coordinates of the minimum bounding rectangle as a reference point, a rectangular region is generated by extending along the film-running direction. The width of the rectangular region is equal to the width of the minimum bounding rectangle, and the height of the rectangular region is a specified multiple of the width of the minimum bounding rectangle. The rectangular area is used as the marker detection area. The center coordinates of the marker detection area overlap with the center coordinates of the smallest bounding rectangle, and the long side of the marker detection area is parallel to the film walking direction.

[0062] In one embodiment, the first region extraction module 604 is specifically used for: The image information is semantically segmented to obtain the first contour information and color information corresponding to the food to be packaged; Based on the pre-configured mapping relationship between color information and channel data, the target channel data is determined from the multi-channel data in the image information, and the target channel data is analyzed based on connected components to obtain the second contour information corresponding to the food to be packaged. The union of the first contour information and the second contour information is taken as the target contour information corresponding to the food to be packaged.

[0063] In one implementation, the second region generation module 606 is specifically used for: Using the geometric center of the mark detection area as the reference origin, offset by a specified distance along the preset food placement direction to obtain the area's starting baseline; Based on the regional starting baseline, a rectangular food detection area is generated. The extension direction of the food detection area is parallel to the film feeding direction and covers the landing point range of the food to be packaged under normal delivery conditions.

[0064] In one implementation, the control module 608 is specifically used for: Based on the relative positional relationship between the target contour information and the food detection area, determine whether the target contour information is located within the food detection area; If not, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag; If so, based on the relative positional relationship between the target contour information and the mark detection area, determine whether to control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag.

[0065] In one implementation, the control module 608 is specifically used for: Determine the overlapping pixel area between the target contour information and the marker detection area; Determine whether the area of ​​overlapping pixels is greater than the second area threshold; If so, then determine that the food to be packaged meets the cutting conditions, and prohibit the control of the sealing bag cutting mechanism from performing cutting actions on the food sealing bag; If not, the control of the sealing bag cutting mechanism will perform a cutting action on the food sealing bag.

[0066] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0067] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A visual inspection method for food sealed bags, characterized in that, This invention relates to a food sealing device, comprising a conveyor belt, a vision inspection mechanism, and a sealing bag cutting mechanism. A continuous series of food sealing bags are placed on the conveyor belt. The outer surface of each food sealing bag is periodically printed with markings indicating cutting positions along the film feeding direction. The film segment between two adjacent markings forms an independent packaging unit. The food to be packaged is placed on the inner surface of the packaging unit. The vision inspection mechanism includes a marking detector, a marking reflection structure, and a vision sensor. When the mark detector detects the mark on the food sealing bag, the visual sensor is triggered to acquire image information of the food sealing bag according to a preset image acquisition delay. The content displayed in the image information includes at least the mark located on the outer surface of the food sealing bag and the food to be packaged located on the inner surface of the food sealing bag. The presentation of the mark in the image information depends on the optical effect of the mark reflection structure. Based on the multi-channel data in the image information, the target contour information corresponding to the food to be packaged and the mark detection area corresponding to the mark are extracted respectively. A food detection area is generated based on the aforementioned marker detection area; Based on the relative positional relationship between the target contour information and the mark detection area and the food detection area, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag.

2. The visual inspection method for food sealed bags according to claim 1, characterized in that, The sign reflection structure includes a prism, at least two symmetrically arranged light sources, and a backlight arranged in sections along the film-moving direction; wherein, The prism is used to allow the optical information of the mark to enter the field of view of the vision sensor via a surface reflection path when the food sealing bag is made of an opaque material. At least two of the light sources project light onto the food sealing bag in a non-normal incidence manner; The backlight is used to illuminate only the section near the logo when the food sealing bag is made of an opaque material.

3. The visual inspection method for food sealed bags according to claim 1, characterized in that, Based on the multi-channel data in the image information, the sign detection region corresponding to the sign is extracted, including: Obtain the region of interest (ROI) from the image information; Based on the region of interest marked, connected component analysis is sequentially performed on the multi-channel data in the image information, and it is determined whether the connected component analysis extracts candidate regions that satisfy preset area constraints. If so, then generate the flag detection region corresponding to the flag based on the center coordinates of the candidate region; If not, then based on the region of interest of the sign, subpixel edge detection is sequentially performed on the multi-channel data in the image information to obtain the subpixel coordinates of the two sides of the sign, and the sign detection region corresponding to the sign is generated according to the subpixel coordinates.

4. The visual inspection method for food sealed bags according to claim 3, characterized in that, Based on the region of interest marked, connected component analysis is sequentially performed on the multi-channel data in the image information, and it is determined whether the connected component analysis extracts candidate regions that satisfy a preset area constraint, including: Extract the current channel data corresponding to the region of interest of the marker from the image information; Perform connected component analysis on the grayscale image corresponding to the current channel data to obtain candidate regions; If the area of ​​the candidate region is less than the first area threshold, it is determined that the preset area constraint is not met, and the next channel data corresponding to the marked region of interest is extracted from the image information. If the area of ​​the candidate region is greater than the first area threshold, it is determined that the preset area constraint is satisfied.

5. The visual inspection method for food sealed bags according to claim 3, characterized in that, Generate the flag detection region corresponding to the flag based on the center coordinates of the candidate region, including: Determine the center coordinates and width of the minimum bounding rectangle of the candidate region; Using the center coordinates of the minimum bounding rectangle as a reference point, a rectangular region is generated by extending along the film-moving direction. The width of the rectangular region is equal to the width of the minimum bounding rectangle, and the height of the rectangular region is a specified multiple of the width of the minimum bounding rectangle. The rectangular region is used as the marker detection region. The center coordinates of the marker detection region overlap with the center coordinates of the minimum bounding rectangle, and the long side of the marker detection region is parallel to the film-moving direction.

6. The visual inspection method for food sealed bags according to claim 1, characterized in that, Based on the multi-channel data in the image information, the target contour information corresponding to the food to be packaged is extracted, including: The image information is semantically segmented to obtain the first contour information and color information corresponding to the food to be packaged; Based on the pre-configured mapping relationship between color information and channel data, target channel data is determined from the multi-channel data in the image information, and connected component analysis is performed on the target channel data to obtain the second contour information corresponding to the food to be packaged. The union of the first contour information and the second contour information is taken as the target contour information corresponding to the food to be packaged.

7. The visual inspection method for food sealed bags according to claim 1, characterized in that, Generate a food detection area based on the marker detection area, including: Using the geometric center of the mark detection area as the reference origin, offset by a specified distance along the preset food placement direction to obtain the area's starting baseline; Based on the starting baseline of the region, a rectangular food detection area is generated. The extension direction of the food detection area is parallel to the film-moving direction and covers the landing point range of the food to be packaged under normal delivery conditions.

8. The visual inspection method for food sealed bags according to claim 1, characterized in that, Based on the relative positional relationship between the target contour information and the mark detection area and the food detection area, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag, including: Based on the relative positional relationship between the target contour information and the food detection area, it is determined whether the target contour information is located within the food detection area; If not, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag; If so, based on the relative positional relationship between the target contour information and the mark detection area, it is determined whether to control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag.

9. The visual inspection method for food sealed bags according to claim 8, characterized in that, Based on the relative positional relationship between the target contour information and the mark detection area, determine whether to control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag, including: Determine the overlapping pixel area between the target contour information and the marker detection area; Determine whether the area of ​​the overlapping pixels is greater than the second area threshold; If so, it is determined that the food to be packaged meets the cutting conditions, and the control of the sealing bag cutting mechanism to perform cutting action on the food sealing bag is prohibited; If not, the sealing bag cutting mechanism is controlled to perform a cutting action on the food sealing bag.

10. A visual inspection device for food sealed bags, characterized in that, This invention relates to a food sealing device, comprising a conveyor belt, a vision inspection mechanism, and a sealing bag cutting mechanism. A continuous series of food sealing bags are placed on the conveyor belt. The outer surface of each food sealing bag is periodically printed with markings indicating cutting positions along the film feeding direction. The film segment between two adjacent markings forms an independent packaging unit. The food to be packaged is placed on the inner surface of the packaging unit. The vision inspection mechanism includes a marking detector, a marking reflection structure, and a vision sensor. An image acquisition module is used to trigger the visual sensor to acquire image information of the food sealing bag according to a preset image acquisition delay when the mark detector detects the mark on the food sealing bag. The content displayed in the image information includes at least the mark located on the outer surface of the food sealing bag and the food to be packaged located on the inner surface of the food sealing bag. The presentation of the mark in the image information depends on the optical effect of the mark reflection structure. The first region extraction module is used to extract the target contour information corresponding to the food to be packaged and the sign detection region corresponding to the sign based on the multi-channel data in the image information. The second region generation module is used to generate a food detection region based on the mark detection region; The control module is used to control the sealing bag cutting mechanism to perform a cutting action on the food sealing bag based on the relative positional relationship between the target contour information and the mark detection area and the food detection area.

Images