A method for detecting a defect image of a printed character of food packaging

Abstract

The present application relates to the technical fields of machine vision and printing quality detection, and discloses a food packaging printing character defect image detection method, a processor reads a to-be-printed string and printing process data, collects character images of the same packaging under different illuminations, a character region and an expanded range are not involved in deformation control point extraction, a local non-rigid deformation field is established by using peripheral packaging structure features and an expected character structure is mapped, a reflection shielding mask is generated according to illumination response differences and reflection probability maps, actual defects are confirmed by combining stroke recovery and character skeleton changes in another illumination image, defect coordinates are converted into printing execution units and time sequence coordinates through reverse mapping of the deformation field, the same kind of defects in continuous packaging repeatedly occur at the same execution position, and when reflection overlap and local deformation are lower than an exclusion threshold, it is determined that there is a persistent printing fault, the method can reduce wrinkle and mirror reflection false detection, and provides a fault tracing basis for inkjet, thermal transfer, laser marking equipment.

Description

Technical Field

[0001] This invention relates to the field of machine vision and printing quality inspection technology, and in particular to a method for detecting defects in printed characters on food packaging. Background Technology

[0002] Production date, shelf life, batch number, and traceability code are usually printed near the packaging seal, on the label edge, or on the curved surface of the bottle. The characters are small, and the production line runs at high speeds. Unstable ink output from the nozzles, insufficient heating in heat transfer printing, or fluctuations in laser energy can all leave behind broken strokes, insufficient ink, diffusion, adhesion, and positional misalignment. There is also a type of situation that is easily confused on-site: the characters themselves are not printed badly, but the soft bag is bulging, the film is wrinkled, or the glossy material reflects highlights, making the captured image look like a stroke is missing. Existing character detection methods mostly start from the image itself. Template matching, grayscale projection, and image difference can find relatively obvious gaps, and affine transformation or perspective correction can also correct the overall skew of the packaging.

[0003] For example, the existing Chinese patent with publication number CN106586135B discloses a method for rejecting defective products with printed production dates on product packaging boxes. This method uses character templates, boundary rectangles, grayscale projection, and detection points to determine whether there are missing strokes in the production date of the packaging box, and rejects defective products accordingly. This type of solution is suitable for packaging surfaces with relatively stable contours, but there is still room for improvement in cases of local bulging of soft bags, uneven stretching of heat shrink film, and strong reflective coverage of strokes.

[0004] The above-mentioned and existing related technologies often have the following drawbacks:

[0005] 1. Mirror reflections change position and intensity depending on the direction of illumination. Although a single image becomes clearer after enhancement, it cannot reliably indicate whether a bright white area is due to highlights obscuring the strokes or whether there is indeed no ink layer or marking mark at that location. Existing systems typically only issue alarms or rejection results after detecting non-conforming packaging. The coordinates of the defective image are not further mapped to the execution positions such as the printhead channel, heating point, or scanning line. When multiple packages have defects in the same location, it is still difficult to distinguish between occasional wrinkles, reflective interference, and persistent equipment failure.

[0006] 2. Deformation on soft bags or heat shrink film is usually not a whole-piece translation. The left side of the character may only be slightly offset, while the right side may be displaced more due to bulging. A single affine or perspective relationship is difficult to describe this local change. If the character outline is directly used as the basis for deformation control, broken strokes, missing ink, and ink diffusion may also participate in the deformation calculation, and the real defects will be partially compensated. If the overall template is still compared, normal deformed characters are easy to be falsely reported. Summary of the Invention

[0007] The technical problem to be solved by the present invention is that the existing technology has the disadvantages of being difficult to adapt to local wrinkles, bulges and non-uniform stretching on the packaging surface, and the specular reflection is easily misjudged as missing strokes. Furthermore, the detected character defects are difficult to correlate with the actual printing execution position, resulting in a high false detection rate of character defects and difficulty in tracing continuous printing failures. To address this, we propose an image detection method for character defects in food packaging printing.

[0008] To achieve the above objectives, this application adopts the following technical solution: a method for detecting defects in printed characters on food packaging, comprising:

[0009] Step 1: The processor reads the string to be printed and the printing process data, and acquires character images of the same package under different lighting conditions;

[0010] Step 2: Set the character area and its surrounding range as the exclusion zone, and establish a local non-rigid deformation field based on the displacement of the control points of the outer packaging structure of the exclusion zone and the continuity of the neighborhood;

[0011] Step 3: Generate the expected character structure diagram based on the string to be printed, and obtain a reference diagram of the deformed character structure using a local non-rigid deformation field;

[0012] Step 4: Generate a reflective occlusion mask based on the differences in illumination response and reflectivity probability map of the character image under different lighting conditions;

[0013] Step 5: Compare the actual character with the string to be printed and the character structure after deformation, and combine the reflective mask, the stroke recovery under another lighting condition, and the changes in the character skeleton to determine the defects of the actual character;

[0014] Step 6: Inversely map the actual character defect coordinates to standard packaging coordinates via a local non-rigid deformation field, and then convert them into printing execution unit coordinates and printing timing coordinates;

[0015] Step 7: Statistically analyze the defect types, character stroke positions, printing execution unit coordinates, and printing timing coordinates in continuous packaging; when the same type of defect repeatedly appears in the same stroke position and is concentrated in the same or adjacent printing execution units, and the degree of reflection overlap and the degree of local deformation are lower than the corresponding exclusion thresholds, a continuous printing failure is determined.

[0016] Step 8: Output a packaging rejection signal based on the actual character defects, and output a printing equipment maintenance prompt based on the continuous printing failure.

[0017] Preferably, the printing process data includes the type of printing equipment, the arrangement of printing execution units, the printing trigger time, the packaging conveying speed, the printing direction, and the calibrated positional relationship between the printing equipment and the image acquisition component; the character image includes a first character image and a second character image acquired by a first illumination unit and a second illumination unit located on different sides of the optical axis of the industrial camera in sequence, and the positional deviation between the two acquisitions is compensated according to the packaging displacement recorded by the conveying encoder.

[0018] Preferably, the packaging structure control points are extracted from one or more structural features, including the sealing line, fixed color block edge, trademark outline, label edge, registration mark, fixed text border, packaging outline, and the intersection of background pattern. A correspondence is established between the standard packaging image and the packaging image to be detected based on the local feature similarity of the candidate control points. Candidate points located in the exclusion area, with insufficient matching relevance, or whose displacement deviates from the neighborhood trend are not included in the deformation field solution. Before this, the reflective candidate area is delineated based on the difference in the initial illumination response of the two character images, and the matching weight of candidate points falling into this area is reduced.

[0019] Preferred, the first The actual displacement vector of the control point of the group packaging structure is given by the first The actual control point and the first The difference between the standard control points is determined; the character region is divided into multiple adjacent grids, and the local non-rigid deformation field is obtained by minimizing the following energy function: ; in, Represents a local non-rigid deformation field The energy function, Represents a local non-rigid deformation field. Indicates the first One standard control point, Indicates the local non-rigid deformation field in The predicted displacement at that location, Indicates the first The actual displacement vector between each standard control point and its corresponding actual control point Indicates the total number of control points. Indicates the total number of adjacent grid pairs. and They represent the first The local displacement vectors of two grids in a pair of adjacent grids. Indicates the displacement continuity constraint coefficient. The L2 norm of a vector. It represents the sum of squares of the components of a vector and is used to measure the squared Euclidean distance between two displacement vectors.

[0020] Preferably, generating the expected character structure diagram includes: decomposing the string to be printed into character units arranged in the printing order; obtaining the expected printing points of each character unit based on the character dot matrix data of the printing device; or converting the character font outline into a character stroke area and extracting the character skeleton to generate an expected character structure diagram including the expected stroke area, character skeleton, skeleton endpoints, skeleton intersections, and character spacing areas; comparing the actual character outline with the deformed character structure diagram to obtain the outline residual; when the outline residual spans multiple characters or is continuously distributed along the packaging fold direction, and the direction of change is consistent with the corresponding local displacement direction, it is determined to be a packaging deformation residual; when the outline residual is limited to a single character or a single stroke and destroys the character skeleton connection relationship, it is determined to be a character defect residual.

[0021] Preferably, generating a reflective occlusion mask includes: performing position registration and brightness normalization on character images under different lighting conditions; determining the lighting response difference based on the brightness difference and gradient difference of corresponding pixels; determining the reflective difference threshold based on the distribution of lighting response difference of background pixels surrounding the character region; identifying pixels that exceed the reflective difference threshold and whose brightness is higher than the corresponding neighborhood brightness in at least one character image as reflective candidate pixels; inputting the character images, brightness difference map, and gradient difference map under different lighting conditions into a reflective region segmentation model to obtain a reflective probability map; and performing cross-validation between reflective candidate pixels and pixels whose reflective probability meets the segmentation conditions; and forming a reflective occlusion mask after merging connected regions, removing isolated regions, and filling internal holes.

[0022] Preferably, the degree of reflective overlap is determined based on the proportion of pixels in the suspected defect area that overlap with the reflective mask to the total number of pixels in the suspected defect area; when the degree of reflective overlap reaches the preset reflective condition, and the suspected missing stroke recovers under another lighting condition and remains connected to the adjacent stroke, it is determined to be reflective masking; when the suspected defect area exists under different lighting conditions, the degree of reflective overlap is lower than the preset reflective condition, and causes an increase in the endpoints of the character skeleton, an interruption of connectivity, or an opening of a closed area, it is determined to be a broken stroke or a lack of ink; when the actual ink layer exceeds the expected stroke area and exists stably under different lighting conditions, it is determined to be ink diffusion; when expected independent characters or strokes form a stable connected area, it is determined to be character adhesion.

[0023] Preferably, a correspondence between standard packaging coordinates and printing process coordinates is established using a calibration graph that includes known printing execution units and known printing timing sequences. The coordinates of the printing execution units correspond to the printhead channel coordinates in inkjet equipment, the row and column coordinates of heating points in thermal transfer equipment, and the scan line coordinates in laser marking equipment. The printing timing coordinates correspond to the printing trigger time, printing pulse sequence number, heating sequence, scanning time, or the relative position of the packaging as it passes through the printing position. When an actual character defect spans multiple printing execution units, the corresponding printing execution unit interval is recorded.

[0024] Preferably, the sliding detection window stores the detection records of 8 to 30 consecutive packages. Normal packages are also included in the window. The detection records are clustered according to defect type, character stroke position, printing execution unit coordinates, and printing timing coordinates. When the same type of broken stroke or missing ink is continuously concentrated in the same or adjacent execution units, corresponding to the same stroke position, and the defect area is relatively stable, it is determined that the execution unit is blocked, has insufficient output, or has an execution abnormality. When the defect position changes randomly and falls on reflective or high-deformation areas, it is treated as occasional imaging interference.

[0025] Preferably, when the overall offset of characters in continuous packaging changes in the same direction, the printing timing coordinates are continuously advanced or delayed, and the packaging structure control points remain stable, it is determined that there may be fluctuations in conveying speed, abnormal printing trigger timing, or deviations in packaging positioning signals. When characters and packaging structures are synchronously offset in similar directions and magnitudes, they are first classified as changes in packaging posture or position, and the timing classification model is only used for auxiliary verification. Only when the probability of continuous printing failure reaches a preset probability threshold and the defect is repeated in the same or adjacent execution units, will a corresponding maintenance prompt be output.

[0026] The technical effects and advantages of this invention are as follows: 1. The response differences under different lighting conditions and the reflectivity probability map together form a reflectivity mask. The stroke recovery and character skeleton changes in another lighting image are then used for verification. The confirmed defects are further mapped to the coordinates of the printing execution unit and the printing timing coordinates. The anomalies that repeatedly occur in the same position in continuous packaging can be further identified as nozzle blockage, insufficient local heating, scanning energy attenuation or trigger timing anomalies. The detection results are used to reject the current packaging and also provide location information for equipment maintenance.

[0027] 2. This method excludes the character area and its outer range from the deformation control point extraction, and only uses the stable outer packaging structure to establish a local non-rigid deformation field. The continuous contour changes caused by wrinkles, bulges and non-uniform stretching can be compensated by the deformation field, while local anomalies such as broken strokes and missing ink will not be absorbed by the deformation calculation. Therefore, it can reduce the situation where normal deformed characters are misjudged as defects. Attached Figure Description

[0028] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts: Figure 1 This is a schematic diagram of the overall process of the food packaging printing character defect image detection method of the present invention; Figure 2 This is a schematic diagram illustrating the process of extracting control points for the packaging structure and establishing a local non-rigid deformation field according to the present invention. Figure 3 This is a diagram showing the expected character structure mapping and character contour residual overlay results of the present invention; Figure 4 This is a flowchart of the reflective mask generation and character joint detection process of the present invention; Figure 5 This is a schematic diagram illustrating the transformation of character defect locations into printing process coordinates according to the present invention; Figure 6 This is a schematic diagram illustrating the statistical distribution of continuous packaging defects in different printing execution units according to the present invention. Detailed Implementation

[0029] The actual processing procedure of this method is described below with reference to the accompanying drawings and embodiments. The numerical values, model forms, and equipment types in the embodiments are used to illustrate the technical relationships. The production line speed, lighting angle, grid density, and judgment threshold can be adjusted according to the packaging material and printing process.

[0030] The data transfer relationships between each step remain unchanged: packaging structure features are used to establish the deformation field, the deformation field is used to correct the expected character structure and recover the defect coordinates; multi-lighting images are used to generate reflective occlusion masks; and defect records that have undergone deformation and reflective verification are then entered into the printing process for coordinate statistics.

[0031] Food packaging can be composite film soft bags, vacuum packaging bags, heat shrink film packaging, paper boxes, plastic cups, plastic bottles, glass bottles, metal cans, or labels attached to the surface of the above packaging. The characters detected include production date, shelf life, product batch number, shift code, and traceability code, which can be formed by inkjet printing, thermal transfer printing, laser marking, or other online marking methods.

[0032] Packaging structural features are located outside the character area and remain relatively stable on packaging of the same size, such as sealing lines, fixed color block edges, trademark outlines, label edges, registration marks, fixed text borders, packaging outlines, and the intersection of background patterns.

[0033] The local non-rigid deformation field describes the direction and amount of displacement of different positions of the packaging from the standard state to the current state. When establishing the deformation field, character strokes, ink dots or character defect areas are not used as control basis to avoid real broken strokes, missing ink or diffusion being incorrectly absorbed into the deformation compensation.

[0034] The expected character structure diagram is generated from the string to be printed, font, and device character parameters. In addition to the stroke area, it also retains structural information such as character skeleton, endpoints, intersections, allowed stroke width, and spacing between adjacent characters.

[0035] The printing process coordinates are used to describe the actual execution position corresponding to the formation of defects. In inkjet equipment, they can represent the printhead channel and printing phase; in thermal transfer equipment, they can represent the heating point row and column and heating sequence; and in laser marking equipment, they can represent the scan line number and scanning time. Different equipment have different names, but they all first restore the defects to the standard packaging coordinates, and then find the corresponding execution unit and timing position according to the calibration relationship.

[0036] Example 1 (refer to) Figure 1 As shown, this embodiment provides a method for detecting defective characters in food packaging printing, which is executed by a processor in conjunction with a production management system, a printing equipment controller, a conveying controller, an image acquisition component, and a rejection mechanism, including steps 1 to 8.

[0037] Step 1: Before the packaged product enters the inspection station, the production management system saves the current product's packaging specifications, the string to be printed, the character font, and the layout information; the printing equipment controller records the equipment type, execution unit arrangement, trigger time, and character formation parameters; the conveyor controller provides the packaging speed and encoder position, and the processor reads this data and associates it with the current packaging serial number.

[0038] The string to be printed can be combined with the production date, product batch, production line number, shift, and traceability code according to preset rules. For example, in “20260701 L03 1426”, “20260701” represents the date, “L03” represents the production line or shift, and “1426” represents the printing time. This string is both the benchmark for character semantic recognition and used to generate the expected character structure.

[0039] The printing process data includes at least the type of printing equipment, the arrangement of execution units, the printing resolution, the trigger time, the packaging conveying speed, the printing direction, and the calibration relationship between the printing equipment, packaging, and image acquisition components. Each set of data is saved together with the corresponding packaging serial number, which facilitates the subsequent tracing of the defect location to the execution unit and printing sequence.

[0040] The character detection station is equipped with an industrial camera, a first illumination unit, and a second illumination unit. The two illumination units are located on different sides of the camera's optical axis. After the packaging positioning sensor or the conveyor encoder sends a trigger signal, the first character image is acquired first, and then the illumination direction is switched to acquire the second character image. The packaging displacement generated between the two exposures is compensated according to the encoder record, so that the two images correspond to the same packaging and the same character area.

[0041] Industrial cameras can use a global shutter. When the character height is about 2.5mm to 5mm, the object resolution can be set to 0.03mm / pixel to 0.12mm / pixel, so that the height of a single character covers no less than 25 pixels. This range is convenient for identifying small broken strokes. The actual parameters are still determined by the character size, lens field of view and production cycle.

[0042] This embodiment uses a packaging structure feature localization model, a reflective region segmentation model, and a character joint recognition model. The former outputs the position and description vector of the character's outer structure, the reflective region segmentation model gives the pixel-level reflective probability, and the character joint recognition model outputs the actual string and candidate defect areas.

[0043] The model input is not limited to the original image. After registration, brightness difference map, gradient difference map, local non-rigid deformation field, expected character structure map and reflective occlusion mask can all be used as additional channels or intermediate features. In this way, character judgment utilizes image appearance, lighting changes, packaging deformation and expected structure at the same time, and draws conclusions without relying on a single image.

[0044] At the end of step 1, the current packaging has the string to be printed, printing process data, two character images, packaging serial number and collection identifier. These data are respectively entered into the deformation estimation, reflection recognition, character comparison and printing coordinate transformation stages.

[0045] Step 2: Determine the character area based on the packaging specifications and layout position. If the packaging has registration marks, label edges, fixed color blocks, or sealing lines, locate these structures first, and then find the character area according to the preset relative position. If there are no independent positioning marks, use the packaging outline and layout coordinates to complete the positioning.

[0046] A packaging structure feature search area is set around the character area, prioritizing the extraction of sealing lines, color block edges, label edges, trademark outlines, and fixed pattern corners. These structures have relatively stable relative positions on packaging of the same size, and will continuously shift with bending, wrinkling, or bulging, making them suitable for reflecting packaging deformation near the characters.

[0047] The character region and its outer extent constitute the character exclusion area. The outer extent distance can be 1 to 3 times the average stroke width. Ink dots, character edges and local textures within the exclusion area do not participate in control point extraction, thus avoiding broken strokes, diffusion or adhesion from affecting deformation estimation.

[0048] The standard packaging image can be derived from the packaging design drawing or established from qualified samples with smooth surfaces and complete characters. The packaging structure feature localization model outputs candidate control point heatmaps and local description vectors from the search areas of the standard packaging image and the image to be detected, respectively.

[0049] Candidate control points are established according to the similarity of description vectors. Candidate points with low matching correlation or whose displacement direction or displacement amount deviates significantly from the overall trend of neighboring points are eliminated. To avoid reverse dependence in subsequent steps, this step only uses the difference in the initial illumination response after coarse registration of the two images to delineate the reflective candidate region. Candidate points falling into this region have their matching weight reduced. Finally, the reflective occlusion mask is still generated in step 4.

[0050] The area near the character is divided into rectangular or triangular grids. Control point displacement is used to constrain the local displacement of each grid. Continuity is maintained between adjacent grids, which suppresses abrupt changes caused by isolated mismatches and preserves continuous local changes caused by wrinkles.

[0051] After the calculation is completed, a local non-rigid deformation field covering the character area is obtained. This deformation field records the displacement direction and displacement amount of each position relative to the standard packaging coordinates, and is then used for the forward mapping of the expected character structure and the reverse recovery of the defect coordinates.

[0052] Step 3: The string to be printed is split into sequentially arranged character units. The character font, character height, spacing, arrangement direction, and device dot matrix parameters jointly determine its standard layout.

[0053] Dot matrix printing equipment can directly read character dot matrices; outline fonts or vector fonts are converted into stroke areas and character skeletons are extracted; thermal transfer patterns and laser scanning trajectories can also be converted into comparable character structures in the same way.

[0054] The resulting expected character structure diagram includes at least the expected stroke area, character skeleton, skeleton endpoints, intersections, and character spacing, and can also record the allowable stroke width, character size, and positional deviation range.

[0055] Then, the deformation field obtained in step 2 is called to map the expected character structure from the standard packaging coordinates to the current image coordinates, and the deformed character structure reference is obtained. Since the character itself does not participate in the selection of control points, the continuous displacement of the packaging can be compensated, and the real defects limited to a single stroke will still be retained between the actual image and the reference.

[0056] Step 4: The first and second character images are first registered using the outer structure of the characters. If there is packaging displacement in the two acquisitions, compensation is first performed according to the record of the conveyor encoder, and then the remaining deviation is corrected by local feature matching.

[0057] After registration, brightness normalization is performed, and the brightness, contrast, and edge changes of the same pixel or local area under two lighting conditions are compared. Specular highlights will change position or intensity with the direction of illumination, while the geometric positions of character ink and marking areas are relatively stable. Based on this, candidate reflective areas can be delineated.

[0058] The reflective region segmentation model receives a first character image, a second character image, a brightness difference map, and a gradient difference map, and outputs a reflective probability map of the same size as the character image. After the reflective candidate pixels are crossed with the pixels that meet the conditions in the probability map, small region removal, connected region merging, and hole filling are performed to obtain a reflective occlusion mask.

[0059] The mask does not delete the original image content; it only indicates which strokes may be temporarily covered by highlights in subsequent judgments.

[0060] Step 5: Compare the actual identified string with the string to be printed character by character. The actual character structure is compared with the deformed character structure with the reference figure, thereby obtaining the suspected defect area and the contour residual.

[0061] The character semantic recognition branch provides the character category and recognition confidence; the structure detection branch extracts stroke regions, skeleton endpoints, intersections, connected regions, stroke widths and character spacing; the defect segmentation branch can output the probability of broken strokes or ink shortages, diffusion, adhesion, positional offsets and normal regions, but this probability map is only a candidate result and needs to be verified by structural and reflective conditions.

[0062] When the semantic content is correct but the stroke connection is interrupted, and the degree of overlap between the abnormal area and the reflective mask is low, it is judged as a broken stroke or lack of ink. When the actual ink layer is stable beyond the expected stroke boundary, it is judged as diffusion. When originally independent characters or strokes form a new stable connected area, it is judged as adhesion.

[0063] When the suspected missing area clearly overlaps with the reflective mask, another illumination image is called for verification. The corresponding strokes in the other image are restored and the original skeleton connection relationship is re-formed, indicating that the gap in the previous image is caused by reflection and is not considered as actual ink loss.

[0064] If the actual string does not match the string to be printed, but the character structure is complete, it is judged as an error in the printed content; if the semantic results are inconsistent and the structure shows obvious breaks, missing or sticking, it is judged as missing or severely damaged characters. The final output includes defective characters, defect type, defect area and image coordinates.

[0065] Step 6: After confirming the defects, first preserve the image coordinates. When there are wrinkles, bulges or surface displacements in the packaging, use the local non-rigid deformation field established in Step 2 to perform inverse mapping and restore it to the standard packaging coordinates.

[0066] After equipment installation, printhead replacement, or packaging specification switching, a set of calibration graphics with known execution units and printing timing is first generated. The camera captures the calibration graphics, and the controller synchronously records the execution unit number and timing information, thereby establishing the correspondence between standard packaging coordinates and printing process coordinates.

[0067] In inkjet equipment, the coordinates of the printing execution unit correspond to the nozzle channel; in thermal transfer equipment, they correspond to the heating point row and column; and in laser marking equipment, they correspond to the scanning line or scanning section. When a defect spans multiple execution units, the coverage area is recorded. The printing timing coordinates are determined based on the trigger time, printing pulse, heating timing, or scanning time.

[0068] Each package generates an inspection record, which includes the package serial number, the string to be printed, the actual string, the defect character, the defect type, the defect image coordinates, the standard package coordinates, the coordinates of the printing execution unit, the printing timing coordinates, the degree of reflection overlap, and the degree of local deformation. This record is then entered into a continuous statistical process.

[0069] Step 7: The inspection records are entered into the sliding inspection window in the order in which the packaging passes through the inspection station, and statistics are performed according to defect type, character stroke position, printing execution unit coordinates, and printing timing coordinates.

[0070] When similar ink shortages or broken strokes repeatedly occur at the same stroke position and are concentrated in the same or adjacent execution units, check the degree of reflection overlap and the degree of local deformation. If both are below the corresponding exclusion threshold, it indicates that the defect is unlikely to be caused by highlights or wrinkles, and it can be determined as a continuous printing failure.

[0071] If the defect location changes randomly, or falls on different reflective, wrinkled, or highly deformable areas each time, it is treated as occasional imaging interference. If the characters as a whole shift in the same direction piece by piece, and the packaging structure position remains stable, then the conveyor speed, printing trigger, and packaging arrival signal are checked.

[0072] Step 8: After confirming the actual character defect, output a rejection signal for the current package. If the sliding window meets the continuous fault conditions, output a separate maintenance prompt for the printing equipment, which includes the suspected execution unit, defect type and recurrence location. If the reflection occlusion is verified as normal stroke by another illumination image, the package will not be rejected and will not be included in the fault statistics.

[0073] Example 2 (refer to) Figure 2 and Figure 3 As shown, this embodiment illustrates the establishment of a local non-rigid deformation field and the separation of packaging deformation residuals and character defect residuals, which is applicable to scenarios where local wrinkles are easily generated, such as composite film soft bags, vacuum packaging bags, and heat shrink film packaging.

[0074] like Figure 2 As shown in (a) and (b), the standard packaging image can be generated by converting the packaging design drawing or established from qualified samples with flat surfaces and complete characters. In both the standard image and the image to be tested, the character region is first determined, and then the structural feature search area is set around it. The character region and its outer range are used as the exclusion area. The character strokes, ink dots and character edges in the area do not participate in the control point extraction.

[0075] For flexible bag packaging, control points can be preferentially selected at the intersection of heat-sealed edges, corners of fixed color blocks, inflection points of trademark outlines, registration marks, and edges of fixed patterns. The positioning model takes the image of the character's outer perimeter as input, and after convolutional feature extraction and multi-scale fusion, it outputs a feature point heatmap and corresponding local description vectors, which can cover both long sealing lines and identify small graphic corners.

[0076] Local maxima are obtained from the heatmap as candidate control points. Then, a correspondence is established based on the similarity of the description vectors between the standard image and the image to be detected. Feature points in the character exclusion area are directly discarded. After coarse registration of the two images, reflective candidate regions are obtained based on the difference in the initial illumination response. The matching weight of candidate points falling into this region is reduced. Here, the candidate regions in step 2 are used. The final reflective occlusion mask formed in step 4 does not participate in the deformation field solution in reverse.

[0077] For the i-th group of control points, the standard control point is denoted as . The actual control point is denoted as Its actual displacement vector Represented as: ; After the correspondence is established, the image correlation, description vector similarity and neighborhood displacement consistency must be checked. If the displacement direction of a certain point is significantly opposite to that of the surrounding points, or the displacement amount is far from the median level of the neighborhood, it will no longer be used for deformation field calculation. This screening does not require the packaging to be flat everywhere, but rather to exclude isolated mismatches while retaining continuous local changes caused by wrinkles.

[0078] The region near the character is divided into multiple adjacent grids, each corresponding to a local displacement vector. The grids can be rectangular or triangular, and their size is determined by the character size, wrinkle width, and control point density: if they are too small, they will easily absorb the individual stroke gaps into the deformation field; if they are too large, they cannot describe non-uniform stretching.

[0079] The local non-rigid deformation field is obtained by minimizing the energy function shown in the following equation: ; in, Represents a local non-rigid deformation field The energy function, where D represents the local non-rigid deformation field. Indicates the first One standard control point, Indicates the deformation field at The predicted displacement at that location, Indicates the first The displacement vector between a standard control point and its corresponding actual control point. Indicates the total number of control points. Indicates the total number of adjacent grid pairs. and They represent the first The local displacement vector of a pair of adjacent grids. Indicates the displacement continuity constraint coefficient. The L2 norm of a vector. It represents the sum of squares of the components of a vector and is used to measure the squared Euclidean distance between two displacement vectors.

[0080] The first term of the energy function ensures that the predicted displacement matches the actual changes at the control point, while the second term avoids unfounded abrupt changes between adjacent grids. Cardboard boxes or rigid labels have high overall stiffness, allowing for the use of larger values. Composite films are prone to forming narrow wrinkles. It can be appropriately reduced to preserve the true local displacement differences.

[0081] After obtaining the deformation field, the structural edges in the standard packaging image are mapped to the image to be detected, and the distance between the mapped edge and the actual edge is checked. If most edges fall within the allowable range, the current result is accepted. If a local deviation is abnormal, the corresponding control point is re-checked, and low confidence or displacement abnormal points are removed before solving again.

[0082] Reference Figure 3 As shown, the confirmed deformation field is used to map the expected stroke area, character skeleton, endpoints, intersections and character intervals to obtain a reference to the deformed character structure. After comparing the actual character outline with the reference, a residual consisting of positional deviation, outline missing and outline overflow is formed.

[0083] When the residual spans multiple characters or is continuously distributed along the fold direction, and the direction of change is basically consistent with the local displacement of the corresponding grid, it is classified as packaging deformation residual and not directly judged as a printing defect.

[0084] When residuals appear only in a single character or within a stroke, and simultaneously cause skeleton breakage, increased endpoints, opening of closed areas, or local ink layer loss, they are classified as character defect residuals. When the actual ink layer exceeds the expected boundary, and the excess portion cannot be explained by the continuous displacement of adjacent grids, it is considered a candidate region for diffusion or adhesion.

[0085] The displacement status of the outer control points is also used for secondary verification. When the character is offset as a whole and the surrounding control points have similar displacement in direction and magnitude, it can be explained by local deformation of the packaging. When a character stroke is partially missing but the outer control points still maintain continuous displacement, the missing stroke cannot be attributed to packaging deformation.

[0086] For example, if the area containing the number "8" is stretched diagonally, and the two closed areas above and below move together with the outer control points, it is an interpretable packaging deformation. If the stroke on the upper right side is missing, but the outer control points still maintain continuous displacement, and the stroke is retained in the reference after deformation, then the gap will generate a new skeleton endpoint, and should continue to be treated as a broken stroke or missing ink.

[0087] Example 3 (refer to) Figure 4 As shown, this embodiment further explains the joint judgment process of reflective occlusion verification, character semantic recognition and character structure detection based on embodiment one or embodiment two. It is mainly used to deal with the occlusion of character strokes caused by the mirror reflection on the surface of metal composite film, glossy plastic packaging and laminated label.

[0088] The first and second illumination units are respectively set on both sides of the optical axis of the industrial camera and adopt different illumination angles. For packaging with strong reflectivity, polarizers can also be set in front of the illumination units and the camera lens to make the specular highlights produce different brightness responses in the two images.

[0089] After the first and second character images are registered in position and normalized in brightness, they are respectively denoted as... and Differences in illumination response at the same pixel location Represented as: ; in, Represents pixel coordinates, A positive number set to prevent the denominator from being zero.

[0090] The processor adaptively determines the reflectivity difference threshold based on the difference in illumination response of the background surrounding the character area. Specifically, it extracts the background area surrounding the character area that does not contain character strokes or fixed color blocks, calculates the illumination response difference value of all pixels in this area, and statistically analyzes its distribution characteristics. The median of the background area response difference is used as the baseline level, and the absolute median difference is used as the measure of dispersion. The threshold is set as the sum of the median and the absolute median difference multiplied by the adjustment coefficient. The adjustment coefficient is selected in the range of 2.5 to 4.5 based on the surface gloss and reflectivity of the packaging material. The smoother the surface and the stronger the reflectivity, the larger the adjustment coefficient value. Through the above adaptive method, it can avoid the missed or false recognition of reflective areas caused by the fixed threshold under different packaging materials, background colors, or ink reflectivity. When the illumination response difference of a pixel is higher than the adaptive threshold, and the brightness of the pixel in at least one image is higher than the brightness of the corresponding neighborhood, it is marked as a reflective candidate pixel. This judgment can exclude differences caused only by dark background textures, ordinary character edges, or low brightness noise.

[0091] The input to the reflective region segmentation model includes a first character image, a second character image, an absolute difference map of the two images, and a gradient difference map. The above data are combined according to the channel direction to form a reflective recognition tensor. The model includes an encoder, a feature fusion layer, and a decoder. The encoder extracts brightness, texture, and edge features at different scales. The feature fusion layer fuses image features and difference features under different lighting conditions. The decoder restores the image resolution and outputs a reflective probability map.

[0092] The reflective region segmentation model can use a U-shaped segmentation network, a feature pyramid network, or a network with the same segmentation function. The model structure itself is not the only limitation, and its output needs to maintain the pixel-level reflective probability corresponding to the input character image.

[0093] The reflective region segmentation model employs end-to-end supervised training. During training, for each set of input images, the model outputs a reflective probability map with the same size as the input image. The training loss consists of two weighted components: the first is a binary cross-entropy loss, used to constrain the pixel-by-pixel classification accuracy between the reflective probability of each pixel and the manually labeled area; the second is a Dice loss, used to measure the overall overlap between the predicted reflective region and the real labeled region, thereby enhancing the model's sensitivity to reflective region boundaries and its adaptability to imbalanced positive and negative samples. The two losses are weighted and summed according to preset weights, serving as the overall optimization objective of the reflective region segmentation model.

[0094] The training samples include images with complete characters but partially covered by highlights, images with no obvious reflections but actual ink shortages, images with both reflections and ink shortages, and images where the packaging texture enters the character area. Images under different lighting conditions are grouped together according to the same packaging and the same character position, and the reflective areas are labeled at the pixel level. When both reflections and actual ink shortages exist at the same position, the two types of areas are labeled separately to avoid the model merging them.

[0095] After the model outputs the reflectivity probability map, the processor identifies pixels with probabilities higher than the segmentation threshold as initial reflective pixels. These pixels are then cross-validated with candidate reflective pixels determined based on differences in illumination response. Only pixels that simultaneously meet both the reflectivity probability condition and the illumination response change condition are written into the reflectivity occlusion mask. This process can reduce the probability of packaging backgrounds, ink diffusion areas, or blank areas of characters being mistakenly identified as reflective areas.

[0096] After the reflective occlusion mask is generated, the processor uses the actual character grayscale image, the expected character structure image, and the reflective occlusion mask as different input channels to form a character joint recognition tensor and input it into the character joint recognition model. This model includes a shared feature extraction network, a character semantic recognition branch, and a character defect segmentation branch.

[0097] The shared feature extraction network extracts local strokes, overall contours, endpoints, intersections, connected regions, and character spacing features of characters. The character semantic recognition branch converts the shared features into feature sequences along the character arrangement direction. Through a bidirectional recurrent network, attention encoding network, or visual Transformer, it outputs the category probability of each character position, the actual recognized string, and the recognition confidence. The character defect segmentation branch outputs probability maps corresponding to broken strokes or missing ink, ink diffusion, character adhesion, character position offset, and no defects.

[0098] The character semantic recognition branch is trained using connection-time classification loss, enabling the model to output variable-length strings end-to-end without requiring character-by-character labeling. The character defect segmentation branch is trained under supervision, outputting pixel-level probability maps for five categories: no defects, broken strokes or missing ink, ink spread, character adhesion, and character position offset. The training loss is composed of a weighted average of multi-class cross-entropy loss and Dice loss. The cross-entropy loss constrains the accuracy of pixel-by-pixel classification, while the Dice loss constrains the spatial overlap between the predicted defect region and the labeled defect region. For broken strokes and missing ink, which are small in area but affect character integrity, higher category weights can be assigned to them in the cross-entropy loss, making the model pay more attention to the recall of small-area defects.

[0099] To ensure that the segmentation results are adapted to the character stroke structure, a structural consistency loss is also set. The structural consistency loss is calculated based on the changes caused by the predicted defect region to the number of character skeleton endpoints, the number of connected regions, and the stroke coverage. When the model predicts that a certain region is a broken stroke or lack of ink, but the region does not cause the expected stroke interruption, endpoint increase, or change in connectivity, the corresponding loss is increased. When the predicted region is outside the expected stroke and causes abnormal connection of adjacent characters, its matching weight with the character adhesion or ink diffusion category is increased.

[0100] The total training loss of the character joint recognition model is a weighted sum of character semantic recognition loss, character defect segmentation loss, and structural consistency loss. Among them, the character semantic recognition branch is optimized using connection-time classification loss, enabling the model to output variable-length strings end-to-end without requiring character-by-character labeling. The character defect segmentation loss is calculated as described above. The structural consistency loss is used to ensure that the segmentation result is consistent with the logical relationship between the character stroke skeleton. When the model predicts that a certain area is a broken stroke or missing ink, but the area does not cause the expected stroke interruption or increase in skeleton endpoints, a corresponding penalty term is added. Conversely, when the predicted area causes abnormal connectivity between adjacent characters, its matching weight with the adhesion category is increased. The three losses are combined according to preset weights and used as the overall training objective of the model.

[0101] After the model completes inference, the processor does not directly use the defect probability map as the final result, but instead combines the character semantic result, the expected character structure map, the reflective occlusion mask, and the local non-rigid deformation field to make a joint judgment.

[0102] For suspected stroke missing areas identified by character structure detection The processor calculates the relationship between this area and the reflective mask. overlap rate :

[0103] in, This indicates the number of pixels in the suspected missing area that overlap with the reflective mask. This indicates the total number of pixels contained in the suspected missing area.

[0104] when If the value is higher than the set value and the suspected missing stroke can be recovered in another lighting image, the processor will determine the area as a reflective occlusion and not as an actual character defect. If the suspected missing area exists in both lighting images, its overlap rate with the reflective occlusion mask is low, and it causes an increase in the character skeleton endpoints, interruption of connectivity, or opening of closed areas, it will be determined as an actual broken stroke or missing ink.

[0105] When the actual ink layer exceeds the expected stroke boundary, and the exceeded area exists stably in both illumination images while the main body of the character skeleton remains intact, it is determined to be ink diffusion. When expected independent characters or strokes form the same stable connected region in the actual image, it is determined to be character adhesion.

[0106] When the output position of the character defect segmentation branch is offset, but the actual offset direction of the character outline is consistent with the predicted direction of the local non-rigid deformation field, and the offset is within the allowable range of the corresponding grid, it is classified as packaging deformation and not as a printing position defect. When the offset of the character outline cannot be explained by the deformation field, and multiple characters show an overall offset in the same direction, it is determined to be a character position offset.

[0107] The semantic results and character structure results are retained separately. The production task requires printing the number "8". The actual character has a short broken stroke on the right side, but the semantic recognition branch still recognizes it as "8". The character structure detection finds that the skeleton endpoints have increased and the expected stroke area has continuous missing parts, so it is still judged as a broken stroke defect. If the highlight covers the right stroke of the number "8", the semantic recognition branch misidentifies it as "6". However, the stroke is restored in another lighting image, and the missing area coincides with the reflective mask. Then the semantic recognition is re-executed and it is not judged as an actual character error.

[0108] The training samples for the character joint recognition model are actually collected from the production line. Each set of samples includes at least a first illumination image, a second illumination image, the string to be printed, the character area position, the reflective area annotation, and the character defect area annotation. The defect areas are labeled according to broken strokes or missing ink, ink diffusion, character adhesion, character position offset, and normal areas. The samples cover different packaging materials, packaging colors, character fonts, printing methods, and packaging deformation states. The data can be expanded by using brightness changes, slight blurring, local noise, scale changes, small angle rotation, and continuous non-rigid transformations. The data expansion is only used to supplement the real samples and does not replace the actual collected defect samples.

[0109] Example 4 (refer to) Figure 5 As shown in the figure, this embodiment illustrates the conversion of defect coordinates from image position to printing process position. The input includes the defect character, defect type, defect area, image coordinates, reflection overlap degree, and local deformation degree confirmed in Embodiment 3.

[0110] like Figure 5 As shown in (a) and (b), after the industrial camera is installed, the printing equipment is adjusted, the actuator is replaced, or the packaging specifications are switched, coordinate calibration is performed first. The printing equipment forms multiple calibration points with known actuator numbers and printing timing. The camera captures the calibration graphics, and the controller synchronously records the nozzle channel, heating point, or scan line number and the corresponding time, thereby establishing the correspondence between the standard packaging coordinates and the printing process coordinates.

[0111] When the packaging surface is relatively flat, the defect coordinates can be directly converted according to the calibration relationship. When there are wrinkles, bulges or curved surface displacements, the standard packaging coordinates are first recovered in reverse using the local non-rigid deformation field of Example 2, and then converted into execution unit coordinates and printing timing coordinates. Even if the defect formed by the same execution unit is in a different position in different packaging images, it will still fall into a similar printing position after recovery.

[0112] For printing execution units arranged approximately equidistantly along the longitudinal direction, let the printing plane coordinates of the center of the defect region be... Its corresponding execution unit number Represented as: ; in, Indicates the reference execution unit number, This represents the vertical coordinate corresponding to the reference execution unit. This indicates the actual distance between adjacent execution units. This indicates that the nearest integer is taken. When the defect area spans multiple execution units, the execution unit interval is recorded according to the start and end positions of the area. When the execution units are not equidistant, calibration table interpolation or coordinate relationships in the device controller are used. This represents the direction coefficient of the execution unit number. As the execution unit number increases with the vertical coordinate,... When the execution unit number decreases as the vertical coordinate increases, .

[0113] When the defect region spans multiple printing execution units, let the start and end coordinates of the defect region in the vertical direction be respectively... and The corresponding execution unit numbers are represented as follows: ; ; And the print execution unit interval corresponding to the defect is recorded as follows: ; For printing execution units arranged at unequal intervals, the longitudinal coordinates of the center of each execution unit are pre-calibrated. The execution unit closest to the center of the defect region is determined according to the following formula: ; Alternatively, the execution unit number can be determined by querying a calibration table or by segmented interpolation based on the correspondence between the execution unit number and the printing plane coordinates stored in the device controller.

[0114] Printing time corresponding to the defect Represented as: ; in, Indicates the time corresponding to the baseline printing position. The horizontal coordinates of the reference position. This indicates the actual conveying speed of the packaging as it passes the printing position. For devices using discrete pulses, it can be further converted into printing pulses, heating pulses, or scanning trigger numbers based on the pulse frequency.

[0115] After the coordinate transformation is completed, each package forms a defect record, which includes the package serial number, inspection time, string to be printed, actual string, defect character, defect type, defect area, image coordinates, standard package coordinates, execution unit coordinates, printing timing coordinates, degree of reflection overlap, and degree of local deformation.

[0116] Defect records are written into a sliding inspection window containing N consecutive packages in the order they pass through the inspection station. N can be 8 to 30. Packages without character defects are also retained as normal records, so that the defect repetition rate is calculated with the number of complete consecutive packages as the denominator.

[0117] Reference Figure 6As shown, the records in the window are clustered according to defect type, character stroke position, printing execution unit coordinates, and printing timing coordinates. The number of valid broken strokes or missing ink in each execution unit, as well as the number of reflections or occasional interferences, are counted separately. When defects from different packages fall into the same or adjacent execution units, and the defect type and stroke position are the same or similar, they are classified into the same target defect set.

[0118] For each target execution unit or adjacent unit interval, count the number of packages with the same defect in the current window and calculate its proportion to the total number of packages in the window as the defect repetition rate. When the repetition rate reaches the preset conditions, such as not less than 40% or 50%, and the average reflective overlap is lower than the reflective exclusion threshold and the local deformation is lower than the deformation exclusion threshold, the execution unit is listed as a suspected continuous fault location.

[0119] The degree of reflection overlap and the degree of local deformation are the exclusion conditions for continuous fault judgment. If either of the two exceeds the corresponding threshold, even if the defect location is temporarily concentrated, the equipment maintenance conclusion will not be output directly, but the record will continue to be retained for subsequent window review.

[0120] When local ink shortages or broken strokes occur continuously at the same execution position, and the defect area is relatively stable, it may correspond to nozzle blockage, insufficient heating, or low scanning energy. When the expected character structure corresponding to a certain execution unit is completely missing in multiple packages, the drive signal or the execution unit itself should be further checked. When the defect locations are randomly scattered and coincide with reflective or highly deformable areas, they should be treated as occasional packaging interference.

[0121] For example, in a window consisting of 12 consecutive packages, 6 packages have ink shortages in the vertical stroke to the right of the number "0" and in the corresponding positions of execution units 18 and 19. The reflective overlap of each defect is less than 0.2, and the degree of local deformation is also within the normal range. Therefore, execution units 18 and 19 are listed as suspected fault locations and a maintenance prompt is output.

[0122] The source of positional offset defects is determined based on the printing timing changes of continuous packaging. When the overall offset of the string increases or decreases with each piece, while the position of the packaging structure features remains stable, there may be printing trigger drift, conveyor speed fluctuations, or packaging positioning signal deviations. When the characters and packaging structure generate offsets of similar direction and magnitude synchronously, they are first attributed to changes in packaging posture or position.

[0123] The temporal classification model can be used to assist in verifying continuous records in a sliding window. The input sequence includes defect type encoding, execution unit coordinates, printing timing coordinates, defect area, reflection overlap degree, and local deformation degree. The output includes the probability of categories such as nozzle blockage, unstable ink output, insufficient local heating, abnormal scanning energy, trigger timing drift, random reflection of packaging, local wrinkles of packaging, and normal state.

[0124] Training samples are derived from historical testing records and are labeled based on equipment maintenance records, printhead cleaning records, thermal transfer component replacement records, laser energy calibration records, packaging rejection records, and manual review results. Samples lacking clear maintenance results or manual review conclusions are not used to mark persistent fault categories but can be saved as normal or observational samples.

[0125] In an feasible configuration, continuous records are first normalized according to the device range, defect types are encoded using one-hot encoding, numerical features are scaled by intervals, the input sequence length is consistent with the sliding detection window, which is 8 to 30, the model uses two layers of gated recurrent units, the number of hidden units is 64 and 32 respectively, and the output of the second layer is connected to a fully connected layer and a Softmax layer to obtain the probability of each fault category.

[0126] Training data is divided into training set, validation set and test set according to production batch, with a ratio of 70%:15%:15%. Data from the same production batch is not allocated across sets. Training uses multi-class cross-entropy loss and Adam optimizer. The initial learning rate can be set to 0.001. Training stops when the validation loss does not decrease after 10 consecutive training cycles. The preset probability threshold is determined by the false positive rate and false negative rate on the validation set. In this embodiment, it is set to 0.70.

[0127] The time-series classification model is only used for auxiliary verification and does not replace the repetitive judgment of the printing process coordinates. Maintenance prompts will only be output when the probability of a continuous printing fault category reaches the preset probability threshold, the defect recurs in the same or adjacent execution units, and the repetition rate of the target position meets the judgment condition. If the probability is high but the defect position is randomly scattered, or the degree of reflection overlap exceeds the exclusion threshold, it will not be judged as a continuous equipment fault.

[0128] Broken pen, insufficient ink, diffusion, or sticking confirmed in a single package will still be treated as defective packaging, but will not immediately escalate to a persistent fault. The record will remain in the sliding window, awaiting subsequent packaging results to participate in the correlation judgment.

[0129] This fifth example illustrates the application of this method in three scenarios: composite film soft bags, heat transfer paper boxes, and laser-marked bottles.

[0130] The composite film food soft bag is 160mm wide and 240mm high. The character area is located at the bottom of the back of the packaging, with a character height of about 3.2mm. The packaging conveying speed is 0.45m / s. After the packaging is filled, it is easy to form irregular bulges. There are also colored background patterns and mirror reflections near the characters.

[0131] The bottom heat-sealed edge, the edge of the fixed color block, and the corner of the back pattern are used as packaging structural features. The character area and its outer range are not included in the control point extraction. After the candidate control points are screened for credibility, a local non-rigid deformation field is established, and then the expected character structure is mapped to the current packaging image.

[0132] The batch number is formed by heat transfer printing on the paper box packaging. When a certain heating point column has poor contact, the same horizontal position of multiple characters will continue to become lighter. After the defect is restored to the standard packaging coordinates by the deformation field, it corresponds to the heating point row and column calibration table. When the same type of defect is concentrated in the same heating point column in the continuous window, and the degree of reflection overlap and local deformation are both lower than the elimination threshold, the heating unit maintenance prompt is output, and the ribbon tension and printing contact status are checked.

[0133] When the right side of the packaging bulges, the right side of the character area can shift upward by about 1.1mm, while the left side only shifts by about 0.3mm. The deformation field compensates for the actual displacement of different areas and does not consider this non-uniform change as a printing position defect. In the first illumination image, the upper right stroke of the number "8" is covered by a highlight, while in the second illumination image, the stroke is complete. The suspected missing area highly overlaps with the reflective mask, so it is treated as a reflective mask.

[0134] If consecutive packages show ink shortage in the vertical stroke to the right of the number "0", and the defects after restoration are concentrated in the 17th and 18th printhead channels with low reflectivity, then a rejection signal will be output to the defective package, and a prompt will be made to check the printhead status of the 17th and 18th channels.

[0135] The production date and batch number are laser-marked on the curved plastic bottle. When the bottle passes through the inspection station, it may rotate at a small angle, and strip-shaped highlights will appear on the surface. The label edge, trademark frame or fixed contour of the bottle are used to determine the character area and establish a local deformation field.

[0136] Two illumination units illuminate the bottle from the left front and right front respectively. The position of the strip-shaped highlight changes with the illumination direction in the two images. When the lower horizontal stroke of the number "2" is only obscured by the highlight in one image, it is treated as a reflective occlusion. If shallow engravings or missing strokes appear on the same scanning line segment of consecutive bottles, and the scanning sequence positions are close, it prompts to check the laser energy and galvanometer synchronization of the scanning segment. When the bottle rotates and the character is synchronously offset with the fixed contour, it is compensated by registration and deformation field, and is not considered as a marking position failure.

[0137] The detection equipment that performs the above method can consist of an industrial computer, an image acquisition component, a printing device controller, a conveying controller, and a rejection mechanism. After the processor executes the program in the memory, it sequentially completes data reading, deformation compensation, reflection verification, character detection, coordinate transformation, and continuous fault judgment.

[0138] The outputs of each model are passed along fixed data relationships: the structural feature localization results are used to establish the deformation field; the reflectivity probability map and the difference in lighting response together form a reflectivity mask; the character semantics and defect probability need to be verified by the expected character structure, reflectivity conditions and local deformation conditions; the fault category probability output by the time series model also needs to be judged together with the repeated distribution of defects in the coordinates of the printing process.

[0139] The model structure can be replaced, but the replaced model should maintain the same type of input and output, and continue to complete character defect confirmation and continuous fault judgment according to the above data relationship.

[0140] The above embodiments are used to illustrate the processing of the present invention. The equivalent substitution of the data relationship between the character exclusion area, the local non-rigid deformation field, the multi-illumination reflection verification, the coordinate transformation of the printing process, and the continuous fault judgment does not change and all fall within the scope of the technical concept of the present invention.

[0141] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A method for detecting defects in printed characters on food packaging, characterized in that, include: S1: The processor reads the string to be printed and the printing process data, and acquires character images of the same package under different lighting conditions; S2: Set the character area and its outer range as the exclusion zone, and establish a local non-rigid deformation field based on the displacement and neighborhood continuity of the control points of the outer packaging structure of the exclusion zone. The packaging structure control points are extracted from one or more packaging structure features, including packaging sealing lines, fixed color block edges, trademark outlines, label edges, registration marks, fixed text borders, packaging outlines, and the intersection of background patterns. A correspondence is established based on the local feature similarity of candidate control points in the standard packaging image and the packaging image to be detected. Candidate control points located within the exclusion area, whose matching correlation is lower than a preset condition, or whose displacement deviates from the displacement trend of neighboring control points are eliminated. Before establishing the local non-rigid deformation field, reflective candidate regions are determined based on the differences in the initial illumination response of the character image under different lighting conditions, and the matching weight of candidate control points falling into the reflective candidate regions is reduced. No. The actual displacement vector of the control point of the group packaging structure is given by the first The actual control point and the first The difference between the standard control points is determined; the character region is divided into multiple adjacent grids, and the local non-rigid deformation field is obtained by minimizing the following energy function: in, Represents a local non-rigid deformation field The energy function, Represents a local non-rigid deformation field. Indicates the first One standard control point, Indicates the local non-rigid deformation field in The predicted displacement at that location, Indicates the first The actual displacement vector between each standard control point and its corresponding actual control point Indicates the total number of control points. Indicates the total number of adjacent grid pairs. and They represent the first The local displacement vectors of two grids in a pair of adjacent grids. Indicates the displacement continuity constraint coefficient. The L2 norm of a vector. This represents the sum of squares of the vector components, used to measure the squared Euclidean distance between two displacement vectors. S3: Generate the expected character structure diagram based on the string to be printed, and obtain the character structure reference diagram after deformation using the local non-rigid deformation field; S4: Generate a reflective occlusion mask based on the differences in the illumination response of the character image under different lighting conditions and the reflectivity probability map; S5: Compare the actual character with the string to be printed and the character structure after deformation, and combine the reflective mask, the stroke recovery under another lighting condition, and the changes in the character skeleton to determine the actual character defects; S6: The actual character defect coordinates are inversely mapped to standard packaging coordinates through the local non-rigid deformation field, and then converted into printing execution unit coordinates and printing timing coordinates; S7: Statistically analyze the defect type, character stroke position, printing execution unit coordinates, and printing timing coordinates in continuous packaging; when the same type of defect repeatedly appears in the same character stroke position and is concentrated in the same or adjacent printing execution units, and the degree of reflection overlap and local deformation of the defect area are lower than the corresponding exclusion thresholds, a continuous printing failure is determined. S8: Output a packaging rejection signal based on the actual character defects, and output a printing equipment maintenance prompt based on the continuous printing failure.

2. The method for detecting defects in printed characters on food packaging according to claim 1, characterized in that: The printing process data includes the type of printing equipment, the arrangement of printing execution units, the printing trigger time, the packaging conveying speed, the printing direction, and the calibrated positional relationship between the printing equipment and the image acquisition component; the character image includes a first character image and a second character image acquired by a first illumination unit and a second illumination unit located on different sides of the optical axis of the industrial camera in sequence, and the positional deviation between the two acquisitions is compensated according to the packaging displacement recorded by the conveying encoder.

3. The method for detecting defects in printed characters on food packaging according to claim 1, characterized in that: Generating the expected character structure diagram includes: decomposing the string to be printed into character units arranged in the printing order, obtaining the expected printing points of each character unit based on the character dot matrix data of the printing device, or converting the character font outline into a character stroke area and extracting the character skeleton to generate an expected character structure diagram including the expected stroke area, character skeleton, skeleton endpoints, skeleton intersections and character spacing areas. The contour residual is obtained by comparing the actual character outline with the deformed character structure with a reference. When the contour residual spans multiple characters or is continuously distributed along the direction of packaging folds, and its direction of change is consistent with the corresponding local displacement direction, it is determined to be a packaging deformation residual. When the contour residual is limited to a single character or a single stroke and destroys the connection relationship of the character skeleton, it is determined to be a character defect residual.

4. The method for detecting defects in printed characters on food packaging according to claim 2, characterized in that: Generating the reflective mask includes: performing position registration and brightness normalization on character images under different lighting conditions; determining the lighting response difference based on the brightness difference and gradient difference of the corresponding pixels; determining the reflective difference threshold based on the distribution of the lighting response difference of the background pixels surrounding the character region; and identifying pixels that exceed the reflective difference threshold and whose brightness is higher than the corresponding neighborhood brightness in at least one character image as reflective candidate pixels. Character images, brightness difference maps, and gradient difference maps under different lighting conditions are input into the reflective region segmentation model to obtain a reflective probability map. The reflective candidate pixels are cross-validated with pixels whose reflective probabilities meet the segmentation conditions. The reflective occlusion mask is formed by merging connected regions, removing isolated regions, and filling internal holes.

5. The method for detecting defects in printed characters on food packaging according to claim 4, characterized in that: The degree of reflective overlap is determined by the proportion of the number of pixels in the suspected defect area that overlap with the reflective mask to the total number of pixels in the suspected defect area. When the degree of reflection overlap reaches the preset reflection condition, and the suspected missing stroke is restored under another lighting condition and remains connected with the adjacent stroke, it is determined to be a reflection blockage; when the suspected defective area exists under different lighting conditions, the degree of reflection overlap is lower than the preset reflection condition, and causes the character skeleton endpoints to increase, the connection relationship to be interrupted or the closed area to open, it is determined to be a broken stroke or lack of ink. When the actual ink layer exceeds the expected stroke area and remains stable under different lighting conditions, it is determined to be ink diffusion; when expected independent characters or strokes form a stable connected area, it is determined to be character adhesion.

6. The method for detecting defects in printed characters on food packaging according to claim 1, characterized in that: By using a calibration graph that includes known printing execution units and known printing timing, the correspondence between the standard packaging coordinates and the printing process coordinates is established; The coordinates of the printing execution unit correspond to the printhead channel coordinates in the inkjet equipment, the row and column coordinates of the heating point in the thermal transfer equipment, and the scan line coordinates in the laser marking equipment; the printing timing coordinates correspond to the printing trigger time, printing pulse number, heating timing, scanning time, or the relative position of the packaging as it passes through the printing position; when the actual character defect spans multiple printing execution units, the corresponding printing execution unit interval is recorded.

7. The method for detecting defects in printed characters on food packaging according to claim 6, characterized in that: A sliding inspection window containing 8 to 30 consecutive packaging inspection records is established. The sliding inspection window is used to save the defect type, character stroke position, printing execution unit coordinates and printing timing coordinates corresponding to each package. Packaging records for which no character defects were detected are also retained in the sliding inspection window as normal samples for continuous defect statistics. Cluster the detection records in the sliding detection window according to the defect type, character stroke position, printing execution unit coordinates, and printing timing coordinates; when broken strokes or missing ink in continuous packaging are concentrated in the same or adjacent printing execution units, corresponding to the same character stroke position, and the defect area is within a preset stable range, it is determined that the corresponding printing execution unit is blocked, has insufficient output, or has an execution abnormality. When defects are randomly distributed in the coordinates of the printing execution unit and coincide with reflective occlusion areas or local high-deformation areas, they are identified as occasional imaging anomalies or packaging surface interference.

8. The method for detecting defects in printed characters on food packaging according to claim 7, characterized in that: When the overall offset of characters in continuous packaging changes in the same direction, the printing timing coordinates are continuously advanced or delayed, and the position of the packaging structure control point remains stable, it is determined that there is a fluctuation in the packaging conveying speed, an abnormal printing trigger timing, or an abnormal packaging arrival signal. When the character position and the packaging structure control point are synchronously offset in the same direction and with similar magnitude, the offset is determined to be a change in packaging posture or packaging position. The defect records arranged in the packaging inspection order in the sliding detection window are input into the time series classification model for auxiliary verification. Only when the probability of the continuous printing failure category output by the time series classification model reaches the preset probability threshold, and the defect appears repeatedly in the same or adjacent printing execution units, the corresponding printing equipment maintenance prompt is output.

Images