Product label inspection system and method for a packaging line

Abstract

The application discloses a product label quality inspection system and method for a packaging line, which comprises a PLC control unit for acquiring target images of inner packaging of bottles or outer packaging of boxes by using photoelectric sensors and industrial cameras and sending the target images to an edge computing gateway; the edge computing gateway is used for acquiring standard label data of a production batch of products to be packaged; a label checking result of the products to be packaged is generated according to the standard label data and the target images in combination with a pre-tuned large language model; when the label checking result indicates that the inner packaging of bottles or the outer packaging of boxes of the products to be packaged is abnormal, a rejection instruction is generated and sent to the PLC control unit; and the PLC control unit is further used for controlling a rejection mechanism to perform a rejection action according to the rejection instruction, so as to push the products to be packaged out of the packaging line. Therefore, the embodiment of the application can adapt to the differentiated quality inspection requirements of multiple varieties and small batches, and significantly improves the universality and flexibility of the system.

Description

Technical Field

[0001] This application relates to the fields of industrial automation and quality inspection technology, and in particular to a product label quality inspection system and method for packaging production lines. Background Technology

[0002] In the chemical manufacturing industry, the production of products such as whitening agents requires both inner bottle packaging and outer box packaging. Each packaging unit must have a label containing crucial information affixed or printed on it. The accuracy of this label information directly affects the product's compliance, the legality of its market circulation, and brand reputation. Therefore, accurately verifying the inner and outer labels of the product is a critical step in ensuring the quality of products leaving the factory.

[0003] In related technologies, standard image templates or fixed text verification rules are pre-designed for the labels of each product. During inspection, the system compares the captured image with the standard template pixel by pixel, or checks whether the extracted text conforms to the preset verification rules, thereby determining whether the label printing is correct.

[0004] However, existing technologies heavily rely on predefined image templates or text validation rules. Once a product model changes, label layout is adjusted, or a new product emerges, technicians must recreate the image templates or text validation rules and adjust the validation algorithm. This process is time-consuming, labor-intensive, and costly to maintain, making it unsuitable for the current flexible manufacturing demands of multi-variety, small-batch, and rapid changeover production. Summary of the Invention

[0005] This application provides a product label quality inspection system for a packaging production line. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general description, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0006] In a first aspect, embodiments of this application provide a product label quality inspection system for a packaging production line, the system comprising:

[0007] The system includes an edge computing gateway, photoelectric sensors integrated into the packaging production line, industrial cameras, a PLC control unit, and a rejection mechanism. The photoelectric sensors include a first photoelectric sensor for detecting the inner packaging of the bottle and a second photoelectric sensor for detecting the outer packaging of the box. The industrial cameras include a first industrial camera for photographing the inner packaging of the bottle and a second industrial camera for photographing the outer packaging of the box. The edge computing gateway communicates with the PLC control unit, which in turn is electrically connected to the photoelectric sensor, industrial camera, and rejection mechanism. The PLC control unit is used to acquire target images of the inner packaging of bottles or the outer packaging of boxes using photoelectric sensors and industrial cameras, and send them to the edge computing gateway. The edge computing gateway is used to acquire standard label data of the production batch of the product to be packaged; based on the standard label data and target image, it generates label verification results of the product to be packaged by combining the pre-fine-tuned large language model; when the label verification results indicate that there is an anomaly in the inner bottle packaging or outer box packaging of the product to be packaged, it generates a rejection instruction and sends it to the PLC control unit. The PLC control unit is also used to control the rejection mechanism to perform rejection actions according to rejection instructions, so as to push the products to be packaged out of the packaging production line.

[0008] Optionally, a photoelectric sensor is used to generate a photo signal and send it to the PLC control unit when the inner bottle packaging or outer box packaging of the product to be packaged arrives at the predetermined photo-taking station. The PLC control unit is also used to send a photo-taking command to the first industrial camera or the second industrial camera based on the photo-taking signal. The first or second industrial camera is used to take pictures of the inner packaging of the bottle or the outer packaging of the box according to the photo-taking instruction, obtain the target image, and send it to the PLC control unit.

[0009] Optionally, based on standard label data and the target image, and combined with a pre-tuned large language model, the label verification results of the product to be packaged are generated, including: Obtain the packaging type of the target image. The packaging type is used to characterize whether the target image belongs to the inner packaging of a bottle or the outer packaging of a box. The packaging type includes the inner packaging type of a bottle or the outer packaging type of a box. Load a preset, dynamically instantiable, generic prompt word template; Based on packaging type and standard label data, the general prompt word template is dynamically instantiated to obtain the target prompt words for the product to be packaged; Input the target prompt words and target image into the pre-fine-tuned large language model to guide the pre-set large language model to check whether there are any anomalies in the target image based on the target prompt words, and obtain the label verification result of the product to be packaged. Output the label verification results of the products to be packaged.

[0010] Optionally, based on packaging type and standard label data, a general prompt word template can be dynamically instantiated to obtain the target prompt words for the product to be packaged, including: Parse the general prompt word template to identify and extract the preset placeholders; the placeholders include packaging type placeholders, standard data placeholders, and validation rule placeholders; Based on the packaging type, obtain the differentiated verification strategy corresponding to the inner packaging type of the bottle or the outer packaging type of the box; Standard label data is serialized into structured text data, and differential verification strategies are transformed into natural language description text. By replacing standard data placeholders with structured text data, replacing packaging type placeholders with packaging type data, and replacing validation rule placeholders with natural language description text, prompt words containing contextual information about the product to be packaged are obtained. Use prompts containing contextual information about the product to be packaged as target prompts for the product to be packaged.

[0011] Optionally, based on packaging type, obtain differentiated verification strategies corresponding to the inner packaging type of bottles or the outer packaging type of boxes, including: Load the local configuration file containing the validation policy rule base, which stores the validation policies associated with the bottle inner packaging type or box outer packaging type; Query the local configuration file to match the set of validation policy descriptions associated with the wrapper type; When the packaging type is bottled inner packaging, extract the first verification rule for bottled inner packaging from the verification strategy description set. The first verification rule includes the font clarity threshold for the production date and batch number, character correctness verification logic, and judgment conditions for whether there are missing or blurred inkjet codes; or, When the packaging type is box-type outer packaging, a second verification rule for box-type outer packaging is extracted from the verification strategy description set. The second verification rule includes the integrity check of the large font information of product name, specifications, and quantity, the readability verification of barcode / QR code, and the location and compliance check of logistics label. The first verification rule can be used as a differentiated verification strategy corresponding to the inner packaging type of bottled products; or the second verification rule can be used as a differentiated verification strategy corresponding to the outer packaging type of boxes.

[0012] Optionally, the target prompt words include the structured text data corresponding to the standard label data in the packaging type placeholder, the natural language description text corresponding to the differentiated verification strategy in the verification rule placeholder, and the packaging type in the packaging type placeholder; Based on the target prompts, check the target image for anomalies to obtain the label verification results of the product to be packaged, including: Based on the packaging type, the target image is analyzed to obtain various visual elements; The structured text data is compared with various visual elements at the character level to verify the consistency between the structured text data and each visual element, and the first verification result is obtained. Using natural language to describe the text, determine whether multiple visual elements are blurred, missing, contaminated, or have their positions shifted beyond a preset offset threshold, and obtain a second verification result; perform existence identification and correctness analysis on the symbols of multiple visual elements, and obtain a third verification result; The first, second, and third verification results are aggregated to obtain the probability of an anomaly of the product to be packaged. When the probability of an anomaly is greater than or equal to a preset decision threshold, a label verification result is generated indicating that there is an anomaly in the inner bottle packaging or outer box packaging of the product to be packaged; or when the probability of an anomaly is less than the preset decision threshold, a label verification result is generated indicating that there is no anomaly in the inner bottle packaging or outer box packaging of the product to be packaged.

[0013] Optionally, the first verification result, the second verification result, and the third verification result are aggregated to obtain the anomaly probability of the product to be packaged, including: The ratio of the number of non-matching characters in the first verification result to the total number of characters is used to obtain the first anomaly score, which is used to characterize the consistency of text content. From the second verification result, the severity of visual defects output by the pre-fine-tuned large language model for each visual element is extracted to obtain a second anomaly score used to characterize visual quality compliance. Based on the existence identification result and correctness analysis result of the identifier in the third verification result, the preset basic defect score is matched to obtain the third anomaly score used to characterize the validity of the identifier. The first, second, and third anomaly scores are weighted and summed to obtain the anomaly probability of the product to be packaged.

[0014] Optionally, a pre-tuned large language model can be generated by following these steps: For each packaging production line, collect historical images of the inner bottle packaging or outer box packaging of different historical packaging products. Obtain the historical standard label data corresponding to each historical image and the historical verification result labels annotated manually. Obtain a pre-trained general vision-language large model as the base model; Based on each historical image and its historical standard label data, generate the model training input corresponding to each historical image; The model training input corresponding to each historical image and the historical verification result label corresponding to each historical image are used as training pairs and input into the base model for iterative training. During training, the underlying parameters of the base model are frozen, and the parameters of the attention mechanism layer and adaptation layer of the base model are updated to minimize the loss value between the predicted verification results output by the base model and the historical verification result labels. When the loss value converges to a minimum, a pre-fine-tuned large language model is generated.

[0015] Optionally, based on each historical image and its historical standard label data, the model training input for each historical image is generated, including: Load a preset, dynamically instantiable, generic prompt word template; Get the packaging type for each historical image; Based on the packaging type of each historical image, obtain a natural language description of the preset differential verification strategy for each historical image; Fill the corresponding placeholders in the general prompt word template with the packaging type, historical standard label data, and natural language description of the preset differential verification strategy for each historical image to obtain the model training prompt words for each historical image. Each historical image is mapped and associated with a model training prompt for that historical image to obtain the model training input for each historical image.

[0016] Secondly, a product label quality inspection method for a packaging production line, the method comprising: The PLC control unit uses photoelectric sensors and industrial cameras to acquire target images of the inner packaging of bottles or the outer packaging of boxes, and sends them to the edge computing gateway. The edge computing gateway obtains the standard label data of the production batch of the product to be packaged; based on the standard label data and target image, and combined with the pre-fine-tuned large language model, it generates the label verification result of the product to be packaged; when the label verification result indicates that there is an abnormality in the inner bottle packaging or outer box packaging of the product to be packaged, it generates a rejection instruction and sends it to the PLC control unit. The PLC control unit controls the rejection mechanism to perform rejection actions according to the rejection command, so as to push the products to be packaged out of the packaging production line.

[0017] In this embodiment, on the one hand, by introducing a pre-tuned large language model, when the product model to be packaged changes, the manager only needs to enter the standard label text data of the corresponding batch into the system. The system can automatically assemble prompts to guide the large language model to verify the product packaging based on the standard label text data and a general prompt word template. This greatly simplifies the system deployment and configuration process, significantly reduces the technical threshold and maintenance workload, and enables the system to quickly respond to the flexible manufacturing needs of multiple varieties and small batches. On the other hand, by deeply integrating photoelectric sensors, industrial cameras, PLC control units, rejection mechanisms, and edge computing gateways, the entire process of unmanned operation, from product arrival detection, automatic image acquisition, real-time AI model analysis to automatic rejection of defective products, is realized. This significantly improves the efficiency of label quality inspection, can adapt to the pace of high-speed production, and promotes the upgrading of packaging production lines towards intelligence and flexibility.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] Figure 1 This is a schematic diagram of the system structure of a product label quality inspection system for a packaging production line provided in an embodiment of this application; Figure 2 This is a schematic diagram of a production line packaging scenario provided in an embodiment of this application; Figure 3 This is a diagram of a system interface for importing standard tag data provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the process of constructing target prompts for a product to be packaged, as provided in an embodiment of this application. Figure 5 This is a flowchart illustrating a product label quality inspection method for a packaging production line provided in an embodiment of this application; Figure 6 This is a flowchart illustrating a method for fine-tuning a large language model provided in an embodiment of this application. Detailed Implementation

[0021] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them.

[0022] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0023] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with some aspects of this application as detailed in the appended claims.

[0024] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0025] Currently, accurate verification of product labels, both inside and out, requires pre-designing standard image templates or fixed text verification rules for each product's label. During inspection, the system compares the captured image with the standard template pixel by pixel, or checks whether the extracted text conforms to the preset verification rules, thereby determining whether the label printing is correct.

[0026] The inventors realized that existing technologies heavily rely on predefined image templates or text validation rules. Whenever a product model changes, label layout is adjusted, or a new product emerges, technicians must recreate the image templates or text validation rules and adjust the validation algorithm. This process is time-consuming, labor-intensive, and costly to maintain, making it unsuitable for the current flexible manufacturing demands of multi-variety, small-batch, and rapid changeover production.

[0027] In this application embodiment, on the one hand, by introducing a pre-tuned large language model, when the product model to be packaged changes, the manager only needs to enter the standard label text data of the corresponding batch into the system. The system can automatically assemble prompts to guide the large language model to verify the product packaging based on the standard label text data and the general prompt word template. This greatly simplifies the system deployment and configuration process, significantly reduces the technical threshold and maintenance workload, and enables the system to quickly respond to the flexible manufacturing needs of multiple varieties and small batches. On the other hand, by deeply integrating photoelectric sensors, industrial cameras, PLC control units, rejection mechanisms, and edge computing gateways, the entire process of unmanned operation from product arrival detection, automatic image acquisition, real-time AI model analysis to automatic rejection of defective products is realized. This significantly improves the efficiency of label quality inspection, can adapt to the pace of high-speed production, and promotes the upgrading of packaging production lines towards intelligence and flexibility. The following exemplary embodiments will be used for detailed description.

[0028] Please see Figure 1 , Figure 1This is a schematic diagram of a product label quality inspection system for a packaging production line provided in an embodiment of this application. The system includes: an edge computing gateway, photoelectric sensors integrated into the packaging production line, an industrial camera, a PLC control unit, and a rejection mechanism; the photoelectric sensors include a first photoelectric sensor for detecting the inner packaging of bottles and a second photoelectric sensor for detecting the outer packaging of boxes; the industrial camera includes a first industrial camera for photographing the inner packaging of bottles and a second industrial camera for photographing the outer packaging of boxes; wherein, the edge computing gateway is communicatively connected to the PLC control unit, and the PLC control unit is electrically connected to the photoelectric sensors, the industrial camera, and the rejection mechanism respectively.

[0029] The edge computing gateway is a distributed computing node deployed on the packaging production line. It is configured with local data storage, preprocessing, and high-performance inference capabilities. Specifically, it receives target image data and production batch information from the PLC control unit, loads and runs a pre-tuned large language model algorithm, and performs multimodal data fusion analysis to generate label verification results or rejection instructions. The edge computing gateway establishes a bidirectional data link with the PLC control unit via a communication bus, enabling the issuance of control commands and the uploading of status data.

[0030] The photoelectric sensor is a position detection device based on the photoelectric effect principle, used for non-contact sensing of the physical presence and position of the product to be packaged. A first photoelectric sensor is positioned on the transport path of the inner bottle packaging and is configured to generate a first electrical signal (i.e., a photo-taking signal) when it detects that the inner bottle packaging has reached a preset first photo-taking trigger position. A second photoelectric sensor is positioned on the transport path of the outer box packaging and is configured to generate a second electrical signal when it detects that the outer box packaging has reached a preset second photo-taking trigger position. The signal output terminals of the photoelectric sensors are electrically connected to the input module of the PLC control unit, serving as a timing reference for triggering subsequent image acquisition actions.

[0031] The industrial camera is an image acquisition device with high frame rate, high resolution, and global shutter characteristics, used to convert light signals into digital image signals. The first industrial camera's field of view covers the label area of ​​the inner bottle packaging and is configured to capture a high-resolution image of the inner bottle packaging in response to a photo-taking command issued by the PLC control unit. The second industrial camera's field of view covers the label area of ​​the outer box packaging and is configured to capture a high-resolution image of the outer box packaging in response to a photo-taking command issued by the PLC control unit. The control interface and data transmission interface of the industrial cameras are electrically connected to the PLC control unit. For example... Figure 2 As shown.

[0032] The PLC control unit serves as the central timing coordination and logic execution core of the entire quality inspection system. Configured to execute a real-time operating system, the PLC control unit receives trigger signals from photoelectric sensors, synchronously schedules industrial cameras for image acquisition, packages the target images acquired from the cameras along with associated production context data, and sends them to the edge computing gateway. It also parses feedback instructions (such as rejection instructions) from the edge computing gateway, thereby driving the actuators. The PLC control unit's inputs are electrically connected to the photoelectric sensors and industrial cameras, its output is electrically connected to the rejection mechanism, and its communication port is connected to the edge computing gateway, forming a closed-loop control circuit.

[0033] The rejection mechanism is a controlled physical actuator used to alter the trajectory or state of non-conforming products on the production line. Upon receiving a rejection command from the PLC control unit, the rejection mechanism executes mechanical actions (such as push rod ejection, air blowing removal, and flip-plate descent) within a preset time window to separate and remove products deemed abnormal from the normal transport flow and exit the packaging line. The drive unit of the rejection mechanism (such as a cylinder solenoid valve or servo motor driver) is electrically connected to the output module of the PLC control unit, receiving discrete or analog control signals.

[0034] Bottled inner packaging is the primary packaging unit that directly contains the product, and its labeling typically includes detailed information such as the production date and batch number in small print. Boxed outer packaging is the secondary transport packaging unit that contains multiple primary packaging units, and its labeling typically includes macroscopic information such as the product name, specifications, and logistics barcode in large print. For example... Figure 2 As shown.

[0035] It should be noted that the inner packaging of the bottle and the outer packaging of the box are treated as differentiated detection objects in the system, corresponding to different sensor detection stations, camera shooting angles, and subsequent differentiated verification strategies for the large language model.

[0036] In some embodiments of this application, the PLC control unit is used to acquire target images of the inner packaging of a bottle or the outer packaging of a box using photoelectric sensors and industrial cameras, and send them to the edge computing gateway; the edge computing gateway is used to acquire standard label data of the production batch of the product to be packaged; generate label verification results of the product to be packaged based on the standard label data, the target image, and a pre-tuned large language model; when the label verification results indicate that there is an anomaly in the inner packaging of the bottle or the outer packaging of the box of the product to be packaged, generate a rejection instruction and send it to the PLC control unit; the PLC control unit is also used to control the rejection mechanism to perform rejection actions according to the rejection instruction, so as to push the product to be packaged out of the packaging production line.

[0037] In some embodiments of this application, a photoelectric sensor is used to generate a photo-taking signal and send it to a PLC control unit when it detects that the inner bottle packaging or outer box packaging of the product to be packaged has arrived at a predetermined photo-taking station; the PLC control unit is also used to send a photo-taking command to a first industrial camera or a second industrial camera according to the photo-taking signal; the first industrial camera or the second industrial camera is used to take a photo of the inner bottle packaging or outer box packaging according to the photo-taking command, obtain a target image, and send it to the PLC control unit.

[0038] The designated camera station is a pre-marked physical inspection area with fixed spatial coordinates along the packaging production line's transport path. The station's position is precisely calibrated so that when the central axis of the product to be packaged (inner packaging for bottles or outer packaging for boxes) or a specific label surface coincides with the geometric center of the station, it is within the optimal depth of field and center of view for the industrial camera. The standard label data for a production batch is an authoritative set of reference information uniquely linked to a specific batch of products currently in production, stored in a database or configuration file. This data set contains all the correct text content (such as product name, specifications, production date code, expiration date, and batch number) and formatting specifications (such as font, font size, and arrangement order) that should be presented on the product label for that batch. The system interface for importing standard label data is as follows: Figure 3 As shown.

[0039] The pre-tuned large language model is a neural network model based on a general multimodal large language model, fine-tuned using a domain-specific historical dataset of packaging defects. This model is configured to possess: visual perception capabilities, enabling it to recognize text, barcodes, and physical defects (blurring, soiling) in images; and logical reasoning capabilities, enabling it to understand the semantic consistency between standard label data and image content, and output structured verification conclusions.

[0040] The label verification result refers to the structured judgment data output by the large language model, which characterizes the quality status of the product to be packaged. This result contains at least a Boolean status indicator (qualified / abnormal), and may further include defect type classification (such as "date error", "missing inkjet code", "position offset"), defect confidence score, and coordinate description of the defect area.

[0041] In one possible implementation, the product to be packaged moves along the conveyor line. When the product enters the detection area before the designated photo-taking station, the corresponding photoelectric sensor (first or second) detects an object obstruction. The photoelectric sensor immediately generates a high-level photo-taking signal and sends it to the PLC control unit. The PLC control unit buffers the received target image and forwards it to the edge computing gateway via a network communication protocol (such as TCP / IP). Simultaneously, the PLC may also send the current production batch number. After receiving the image, the edge computing gateway retrieves the standard label data for that batch from its local database or MES system based on the batch number. The edge computing gateway encapsulates the target image, standard label data, and preset verification rule prompts to construct an input vector for a large language model. The edge computing gateway feeds the input vector into the pre-tuned large language model. The model executes the following inference logic: extracting visual elements (text, barcode) from the image; comparing the extracted content with the standard label data at the character and semantic levels; and evaluating visual quality (clarity, completeness). The model outputs the label verification result (e.g., {Status: “Fail”, Reason: “Date Mismatch”, Confidence: 0.98}). The edge computing gateway parses the verification result. If the result indicates an anomaly, the edge computing gateway immediately generates a rejection instruction and sends it back to the PLC control unit. Upon receiving the instruction, the PLC control unit initiates position tracking logic (e.g., using encoder counting) to monitor the position of the abnormal product on the production line in real time. When the abnormal product reaches the effective range of the rejection mechanism, the PLC outputs a control signal to drive the rejection mechanism to act (e.g., push rod extension, air blower activation). The abnormal product is physically removed from the packaging production line, completing the quality inspection closed loop.

[0042] For example, a food factory produces boxed milk with a production date of "March 23, 2026". Standard label data specifies that the production date must be printed as "20260323", and the font must be clear and without ink gaps. A box of milk on the production line has a malfunctioning inkjet printer, resulting in the date being printed as "20260328" (date error), and a slight ink gap in the lower left corner of the number "8" (visual defect). As this box of milk flows through the production line, a second photoelectric sensor (for the carton / box) detects it and sends a photo signal to the PLC control unit. The PLC control unit triggers a second industrial camera after a 50-millisecond delay. The industrial camera captures an image of the side of the milk box, clearly showing the incorrect date "20260328" and the flawed "8". The PLC control unit transmits the image to an edge computing gateway. The edge computing gateway reads the standard label data as the following JSON data: {"Expected_Date":"20260323"}. The edge computing gateway invokes a pre-tuned large language model, inputting an image and standard data. The large language model first performs semantic comparison: it identifies the text in the image as "20260328," which does not match the standard "20260323," thus indicating a logical anomaly. The large language model then performs visual quality inspection: it identifies a missing stroke in the digit "8," thus indicating a visual anomaly. The large language model generates a label verification result: {"Result":"NG", "Error_Type":["Date_Error", "Ink_Missing"], "Confidence":0.99}. The edge computing gateway determines that the milk carton is abnormal and sends a rejection command to the PLC control unit. The PLC control unit tracks the milk carton, and when it reaches the rejection station, it controls the rejection mechanism (pneumatic pusher). The milk carton with the incorrect date and unclear printing is pushed into the waste bin, while subsequent qualified products pass through normally.

[0043] In some embodiments of this application, the specific process of generating label verification results for a product to be packaged based on standard label data, a target image, and a pre-tuned large language model includes: obtaining the packaging type of the target image, where the packaging type is used to characterize whether the target image belongs to inner bottle packaging or outer box packaging, and the packaging type includes inner bottle packaging or outer box packaging; loading a preset, dynamically instantiable general prompt word template; dynamically instantiating the general prompt word template according to the packaging type and standard label data to obtain the target prompt word for the product to be packaged; inputting the target prompt word and the target image into the pre-tuned large language model to guide the preset large language model to verify whether the target image has any anomalies based on the target prompt word, thereby obtaining the label verification result for the product to be packaged; and outputting the label verification result for the product to be packaged.

[0044] Packaging type is a classification identifier used to characterize the physical hierarchy and visual features of the object to be inspected. It includes at least two types: inner bottle packaging (corresponding to primary packaging, characterized by a small field of view, high detail density, and focus on coding / labels) and outer box packaging (corresponding to secondary transport packaging, characterized by a large field of view, macro layout, and focus on logistics labels / box printing). Packaging type is used to determine which specific inspection logic should be applied subsequently, the definition of the Region of Interest (ROI), and the defect judgment criteria, preventing confusion in inspection strategies for different levels of packaging.

[0045] The preset, dynamically instantiable generic prompt word template is a natural language instruction framework stored in the edge computing gateway, containing a fixed structural framework and variable parameter placeholders. It defines the role settings of the large language model (e.g., you are a senior quality inspector), task objectives (perform tag validation), output format specifications (output JSON format), and general inference logic. Interfaces for populating specific business data are reserved, such as slots for {Packaging_Type}, {Standard_Data}, and {Specific_Rules}.

[0046] The target prompt is a final input instruction sequence containing complete task context information, generated through dynamic instantiation. This sequence integrates metadata descriptions of the visual input (target image), business constraints (standard label data), and a dedicated detection strategy for the current packaging type.

[0047] In one possible implementation, the system receives a target image and its metadata from the preceding process. The sensor source or image tag in the metadata is parsed: if the image originates from a first industrial camera or is labeled Line_A_Bottle, the packaging type is determined to be inner bottle packaging. If the image originates from a second industrial camera or is labeled Line_B_Box, the packaging type is determined to be outer box packaging. The edge computing gateway reads a preset, dynamically instantiable, generic prompt word template from its local storage. An example template structure is as follows: "You are a packaging quality control expert. Please check the product label of type {Packaging_Type}."

[0048] The standard data that must be followed is: {Standard_Data}.

[0049] The specific detection rules for {Packaging_Type} are: {Specific_Rules}.

[0050] Please analyze the input image. If any discrepancies with the standard data or violations of specificity rules are found, mark it as abnormal and explain the reason; otherwise, mark it as qualified.

[0051] Please output in JSON format: {{"status": "...", "defect": "...", "reason":"..."}}.

[0052] Then, for the "bottled inner packaging type," the rule is set as: "Focus on checking the edge sharpness and ink breakage of the inkjet characters, and verify the production date and batch number." For the "boxed outer packaging type," the rule is set as: "Focus on checking the label affixing position offset, barcode integrity, and large font product name spelling." The system obtains the standard label data for the current batch (e.g., {"Date": "20260323", "Batch": "A01"}). Dynamic instantiation is performed, filling the above variables into the template placeholders. {Packaging_Type} corresponds to "bottled inner packaging," {Standard_Data} corresponds to "Production date: 20260323, Batch: A01," and {Specific_Rules} corresponds to "Focus on checking the edge sharpness of the inkjet characters...", ultimately generating a complete target prompt. The target prompt (text) and target image (visual data) are then input into a pre-fine-tuned large language model. The large model is guided by specific detection rules in the target prompts, adjusting its attention mechanism: for bottled goods, the model focuses on pixel-level details of tiny characters; for boxed goods, the model focuses on the overall layout and macroscopic text. The model executes comparison logic to determine whether the image content conforms to standard data and specific rules. The model generates structured label verification results (such as JSON strings). The system parses the results, extracts status indicators (pass / fail), and passes them to subsequent rejection control logic.

[0053] In some embodiments of this application, the specific process of dynamically instantiating a general prompt word template based on packaging type and standard label data to obtain the target prompt word for the product to be packaged includes: parsing the general prompt word template to identify and extract preset placeholders; the placeholders include packaging type placeholders, standard data placeholders, and verification rule placeholders; obtaining differentiated verification strategies corresponding to the inner packaging type of bottles or the outer packaging type of boxes based on the packaging type; serializing the standard label data into structured text data and converting the differentiated verification strategies into natural language description text; replacing the standard data placeholders with structured text data, replacing the packaging type placeholders with packaging type data, and replacing the verification rule placeholders with natural language description text to obtain prompt words containing contextual information about the product to be packaged; and using the prompt words containing contextual information about the product to be packaged as the target prompt words for the product to be packaged.

[0054] Specifically, the process of obtaining differentiated verification strategies corresponding to bottled inner packaging or boxed outer packaging based on packaging type includes: loading a local configuration file containing a verification strategy rule base, which stores verification strategies associated with bottled inner packaging or boxed outer packaging; querying the local configuration file to match the verification strategy description set associated with the packaging type; when the packaging type is bottled inner packaging, extracting a first verification rule for bottled inner packaging from the verification strategy description set, which includes font clarity thresholds for production date and batch number, character correctness verification logic, and judgment conditions for missing or blurred inkjet printing; or, when the packaging type is boxed outer packaging, extracting a second verification rule for boxed outer packaging from the verification strategy description set, which includes integrity checks of large font information for product name, specifications, and quantity, readability verification of barcodes / QR codes, and location and compliance checks of logistics markings; using the first verification rule as the differentiated verification strategy corresponding to the bottled inner packaging type; or using the second verification rule as the differentiated verification strategy corresponding to the boxed outer packaging type.

[0055] For example Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the process of constructing target prompt words for a product to be packaged, as provided in this application. First, the system acquires the packaging type (bottled inner packaging or boxed outer packaging) and serialized standard label data of the target image, and loads a preset general prompt word template. Next, the system parses the template to identify three key placeholders (packaging type, standard data, and verification rules), and queries the differentiated verification strategy rule library in the local configuration file based on the packaging type. If it is bottled inner packaging, the system extracts the first verification rule focusing on "character clarity threshold, correctness verification logic, and inkjet missing / fuzzy judgment"; if it is boxed outer packaging, the system extracts the second verification rule focusing on "large font integrity, barcode readability, and logistics label compliance". Finally, the system converts the extracted specific verification strategies into natural language descriptions, and fills them, along with structured standard data and packaging type identifiers, into the placeholders corresponding to the general template. The system dynamically instantiates and generates target prompt words containing complete contextual information and outputs them, thereby ensuring that the large language model can perform the most suitable quality inspection reasoning task for different packaging forms.

[0056] The target prompts include the structured text data corresponding to the standard label data in the packaging type placeholder, the natural language description text corresponding to the differentiated verification strategy in the verification rule placeholder, and the packaging type in the packaging type placeholder.

[0057] In some embodiments of this application, the specific process of verifying whether an anomaly exists in a target image based on target prompt words to obtain the label verification result of the product to be packaged includes: parsing the target image according to the packaging type to obtain multiple visual elements; performing character-level comparison between structured text data and multiple visual elements to verify the consistency between the structured text data and each visual element to obtain a first verification result; using natural language description text to determine whether multiple visual elements are blurred, missing, contaminated, or have a positional offset exceeding a preset offset threshold to obtain a second verification result; performing existence identification and correctness analysis on the identifiers of multiple visual elements to obtain a third verification result; aggregating the first verification result, the second verification result, and the third verification result to obtain the anomaly probability of the product to be packaged; when the anomaly probability is greater than or equal to a preset decision threshold, generating a label verification result indicating that the inner bottle packaging or outer box packaging of the product to be packaged has an anomaly; or when the anomaly probability is less than the preset decision threshold, generating a label verification result indicating that the inner bottle packaging or outer box packaging of the product to be packaged does not have an anomaly.

[0058] Among them, various visual elements are the smallest semantically independent visual units extracted from the target image through image segmentation or object detection algorithms. This set includes at least: text region elements (such as production date strings, batch number character sequences), graphic symbol elements (such as barcodes, QR codes, certification icons, recycling symbols), and background texture elements (used to assess contamination or background integrity).

[0059] In one possible implementation, the system determines the target image and its packaging type. It then invokes the visual backbone network of a large language model to perform region extraction and segmentation on the image. This extracts various visual elements, including text blocks. (Corresponding date, batch, etc.) Symbol block (Corresponding to barcodes, logos, etc.). Record the coordinates (x, y, w, h) and image slices of each element. The system executes the following sub-processes in parallel. Sub-process A (Content Consistency Verification): Use OCR technology to convert text blocks... Convert to machine text. Perform a character-level comparison between the machine text and structured text data (standard data). If any character mismatch is found (e.g., identified as a comma, or a missing character), mark the error count. Output the first verification result. (e.g., error rate 0.0 or 1.0, or edit distance score). Subprocess B (Visual Quality and Position Verification): Utilizes the visual understanding capabilities of a large language model or multiple traditional image processing operators to analyze image slices of each element. Evaluation is based on natural language descriptions (e.g., "Check for blurriness, missing elements, contamination"). The difference between the actual coordinates and standard coordinates of each element is calculated. ,judge Check if it exceeds the preset offset threshold. Output the second verification result. (For example: a quality score of 0-1, or a list of Boolean symbols). For symbol blocks Perform a decoding attempt (e.g., barcode scanning) or feature matching. Perform presence detection (whether a symbol was detected) and correctness analysis (whether decoding was successful / feature matching degree). Output the third verification result. (e.g., symbol integrity rating).

[0060] Finally, the system will The input is fed into the aggregation engine, and a weighted formula is applied to calculate the anomaly probability. : ; calculate the With preset decision threshold Compare. If : Generate abnormal tag verification results (triggering removal). If Generate a normal label verification result (allow passage). Output the final result to the control unit.

[0061] Specifically, the process of aggregating the first, second, and third verification results to obtain the anomaly probability of the product to be packaged includes: calculating the ratio of the number of mismatched characters to the total number of characters in the first verification result to obtain a first anomaly score representing the consistency of the text content; extracting the severity of visual defects output by the pre-tuned large language model for each visual element from the second verification result to obtain a second anomaly score representing visual quality compliance; matching the existence recognition result and correctness analysis result of the identifier in the third verification result with preset basic defect scores to obtain a third anomaly score representing the validity of the identifier; and weighted summing the first, second, and third anomaly scores to obtain the anomaly probability of the product to be packaged.

[0062] In some embodiments of this application, the specific process of generating a pre-fine-tuned large language model includes: collecting each historical image of the inner bottle packaging or outer box packaging of different historical packaging products for each packaging production line; obtaining historical standard label data and manually annotated historical verification result labels corresponding to each historical image; obtaining a pre-trained general vision-language large model as the base model; generating model training input corresponding to each historical image based on each historical image and the historical standard label data of each historical image; using the model training input corresponding to each historical image and the historical verification result labels corresponding to each historical image as training pairs, and inputting them into the base model for iterative training; during the training process, freezing the underlying parameters of the base model, and updating the parameters of the attention mechanism layer and adaptation layer of the base model to minimize the loss value between the predicted verification result output by the base model and the historical verification result labels; and generating a pre-fine-tuned large language model when the loss value converges to the minimum.

[0063] Specifically, the process of generating the model training input for each historical image based on its historical standard label data includes: loading a preset, dynamically instantiable general prompt word template; obtaining the packaging type of each historical image; obtaining a natural language description of a preset differential verification strategy for each historical image based on its packaging type; filling the corresponding placeholders in the general prompt word template with the packaging type, historical standard label data, and natural language description of the preset differential verification strategy for each historical image to obtain the model training prompt word for each historical image; and mapping and associating each historical image with its model training prompt word to obtain the model training input for each historical image.

[0064] In this embodiment, on the one hand, by introducing a pre-tuned large language model, when the product model to be packaged changes, the manager only needs to enter the standard label text data of the corresponding batch into the system. The system can automatically assemble prompts to guide the large language model to verify the product packaging based on the standard label text data and a general prompt word template. This greatly simplifies the system deployment and configuration process, significantly reduces the technical threshold and maintenance workload, and enables the system to quickly respond to the flexible manufacturing needs of multiple varieties and small batches. On the other hand, by deeply integrating photoelectric sensors, industrial cameras, PLC control units, rejection mechanisms, and edge computing gateways, the entire process of unmanned operation, from product arrival detection, automatic image acquisition, real-time AI model analysis to automatic rejection of defective products, is realized. This significantly improves the efficiency of label quality inspection, can adapt to the pace of high-speed production, and promotes the upgrading of packaging production lines towards intelligence and flexibility.

[0065] Please see Figure 5This application provides a flowchart illustrating a product label quality inspection method for a packaging production line. Figure 5 As shown, the detection method in this application embodiment may include the following steps: S101, the PLC control unit uses photoelectric sensors and industrial cameras to acquire target images of bottle inner packaging or box outer packaging, and sends them to the edge computing gateway; In some embodiments of this application, the photoelectric sensor is used to generate a photographing signal and send it to the PLC control unit when it detects that the inner bottle packaging or the outer box packaging of the product to be packaged has arrived at a predetermined photographing station; the PLC control unit is also used to send a photographing command to the first industrial camera or the second industrial camera according to the photographing signal; the first industrial camera or the second industrial camera is used to take a photograph of the inner bottle packaging or the outer box packaging according to the photographing command, obtain a target image, and send it to the PLC control unit.

[0066] S102, the edge computing gateway obtains the standard label data of the production batch of the product to be packaged; based on the standard label data and target image, and combined with the pre-fine-tuned large language model, it generates the label verification result of the product to be packaged; when the label verification result indicates that there is an abnormality in the inner bottle packaging or outer box packaging of the product to be packaged, it generates a rejection instruction and sends it to the PLC control unit. S103, the PLC control unit controls the rejection mechanism to perform rejection actions according to the rejection command, so as to push the product to be packaged out of the packaging production line.

[0067] In this embodiment, on the one hand, by introducing a pre-tuned large language model, when the product model to be packaged changes, the manager only needs to enter the standard label text data of the corresponding batch into the system. The system can automatically assemble prompts to guide the large language model to verify the product packaging based on the standard label text data and a general prompt word template. This greatly simplifies the system deployment and configuration process, significantly reduces the technical threshold and maintenance workload, and enables the system to quickly respond to the flexible manufacturing needs of multiple varieties and small batches. On the other hand, by deeply integrating photoelectric sensors, industrial cameras, PLC control units, rejection mechanisms, and edge computing gateways, the entire process of unmanned operation, from product arrival detection, automatic image acquisition, real-time AI model analysis to automatic rejection of defective products, is realized. This significantly improves the efficiency of label quality inspection, can adapt to the pace of high-speed production, and promotes the upgrading of packaging production lines towards intelligence and flexibility.

[0068] Please see Figure 6 This document provides a flowchart illustrating a method for fine-tuning a large language model, as illustrated in an embodiment of this application. Figure 6 As shown, the detection method in this application embodiment may include the following steps: S201, for each packaging production line, collect each historical image of the inner bottle packaging or outer box packaging of different historical packaging products; S202, obtain the historical standard label data and manually annotated historical verification result labels corresponding to each historical image; S203, Obtain a pre-trained general vision-language large model as the base model; S204, Generate the model training input corresponding to each historical image based on each historical image and the historical standard label data of each historical image; In some embodiments of this application, the specific process of generating the model training input corresponding to each historical image based on each historical image and its historical standard label data includes: loading a preset, dynamically instantiable general prompt word template; obtaining the packaging type of each historical image; obtaining a natural language description of a preset differential verification strategy for each historical image based on its packaging type; filling the packaging type, historical standard label data, and natural language description of the preset differential verification strategy of each historical image into the corresponding placeholders of the general prompt word template to obtain the model training prompt word for each historical image; and mapping and associating each historical image with its model training prompt word to obtain the model training input corresponding to each historical image.

[0069] S205, take the model training input corresponding to each historical image and the historical verification result label corresponding to each historical image as training pairs, and input them into the base model for iterative training; S206. During training, the underlying parameters of the base model are frozen, and the parameters of the attention mechanism layer and adaptation layer of the base model are updated to minimize the loss value between the predicted verification result output by the base model and the historical verification result label. S207: When the loss value converges to the minimum, a pre-fine-tuned large language model is generated.

[0070] In some embodiments of this application, if the loss value has not converged to the minimum, the step of iterative training in the input base model continues until the loss value converges to the minimum.

[0071] In this embodiment, on the one hand, by introducing a pre-tuned large language model, when the product model to be packaged changes, the manager only needs to enter the standard label text data of the corresponding batch into the system. The system can automatically assemble prompts to guide the large language model to verify the product packaging based on the standard label text data and a general prompt word template. This greatly simplifies the system deployment and configuration process, significantly reduces the technical threshold and maintenance workload, and enables the system to quickly respond to the flexible manufacturing needs of multiple varieties and small batches. On the other hand, by deeply integrating photoelectric sensors, industrial cameras, PLC control units, rejection mechanisms, and edge computing gateways, the entire process of unmanned operation, from product arrival detection, automatic image acquisition, real-time AI model analysis to automatic rejection of defective products, is realized. This significantly improves the efficiency of label quality inspection, can adapt to the pace of high-speed production, and promotes the upgrading of packaging production lines towards intelligence and flexibility.

[0072] This application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the product label quality inspection method for a packaging production line provided in the above-described method embodiments.

[0073] This application also provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the product label quality inspection method for a packaging production line described in the various method embodiments above.

[0074] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for product label quality inspection on a packaging production line can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program for product label quality inspection on a packaging production line can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0075] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A product label quality inspection system for a packaging production line, characterized in that, The system includes: The system includes an edge computing gateway, photoelectric sensors integrated into the packaging production line, industrial cameras, a PLC control unit, and a rejection mechanism. The photoelectric sensors include a first photoelectric sensor for detecting the inner packaging of the bottle and a second photoelectric sensor for detecting the outer packaging of the box. The industrial cameras include a first industrial camera for photographing the inner packaging of the bottle and a second industrial camera for photographing the outer packaging of the box. The edge computing gateway is communicatively connected to the PLC control unit, and the PLC control unit is electrically connected to the photoelectric sensor, industrial camera, and rejection mechanism, respectively; wherein... The PLC control unit is used to acquire target images of the inner packaging of the bottle or the outer packaging of the box using the photoelectric sensor and industrial camera, and send them to the edge computing gateway; The edge computing gateway is used to acquire standard label data of the production batch of the product to be packaged; based on the standard label data and the target image, and combined with a pre-tuned large language model, it generates a label verification result for the product to be packaged, including: acquiring the packaging type of the target image, where the packaging type is used to characterize whether the target image belongs to the inner packaging of the bottle or the outer packaging of the box, and the packaging type includes either the inner packaging type of the bottle or the outer packaging type of the box; loading a preset, dynamically instantiable general prompt word template; dynamically instantiating the general prompt word template based on the packaging type and the standard label data to obtain the target prompt word for the product to be packaged; inputting the target prompt word and the target image into the pre-tuned large language model to guide the preset large language model to verify whether the target image has any anomalies based on the target prompt word, thereby obtaining the label verification result for the product to be packaged; outputting the label verification result for the product to be packaged; and generating a rejection instruction and sending it to the PLC control unit when the label verification result indicates that the inner packaging of the bottle or the outer packaging of the box of the product to be packaged has an anomaly. The PLC control unit is also used to control the rejection mechanism to perform rejection actions according to the rejection instruction, so as to push the product to be packaged out of the packaging production line.

2. The system according to claim 1, characterized in that, The photoelectric sensor is used to generate a photo-taking signal and send it to the PLC control unit when it detects that the inner bottle packaging or the outer box packaging of the product to be packaged has arrived at the predetermined photo-taking station. The PLC control unit is also used to send a photo-taking command to the first industrial camera or the second industrial camera according to the photo-taking signal; The first industrial camera or the second industrial camera is used to take pictures of the inner packaging of the bottle or the outer packaging of the box according to the photo-taking instruction, obtain the target image, and send it to the PLC control unit.

3. The system according to claim 1, characterized in that, The step of dynamically instantiating the general prompt word template based on the packaging type and the standard label data to obtain the target prompt word for the product to be packaged includes: The general prompt word template is parsed to identify and extract preset placeholders; the placeholders include packaging type placeholders, standard data placeholders, and validation rule placeholders; Based on the packaging type, obtain the differential verification strategy corresponding to the inner packaging type of the bottle or the outer packaging type of the box; The standard label data is serialized into structured text data, and the differential verification strategy is converted into natural language description text. The standard data placeholders are replaced with the structured text data, the packaging type placeholders are replaced with the packaging type, and the validation rule placeholders are replaced with the natural language description text, to obtain prompt words containing contextual information about the product to be packaged; The prompt word containing the contextual information of the product to be packaged is used as the target prompt word for the product to be packaged.

4. The system according to claim 3, characterized in that, The step of obtaining a differential verification strategy corresponding to the inner packaging type of the bottle or the outer packaging type of the box based on the packaging type includes: Load a local configuration file containing a rule base for verification strategies, wherein the local configuration file stores verification strategies associated with the inner packaging type of the bottle or the outer packaging type of the box; Query the local configuration file to match the set of verification policy descriptions associated with the packaging type; When the packaging type is the bottled inner packaging type, a first verification rule for the bottled inner packaging is extracted from the verification strategy description set. The first verification rule includes the font clarity threshold of the production date and batch number, character correctness verification logic, and a judgment condition for whether there is missing or blurred inkjet printing; or, When the packaging type is a box-type outer packaging, a second verification rule for the box-type outer packaging is extracted from the verification strategy description set. The second verification rule includes the integrity check of the large font information of product name, specifications, and quantity, the readability verification of barcode / QR code, and the location and compliance check of logistics identification. The first verification rule can be used as a differentiated verification strategy corresponding to the inner packaging type of the bottle; or the second verification rule can be used as a differentiated verification strategy corresponding to the outer packaging type of the box.

5. The system according to claim 1, characterized in that, The target prompt words include the structured text data corresponding to the standard label data in the packaging type placeholder, the natural language description text corresponding to the differentiated verification strategy in the verification rule placeholder, and the packaging type in the packaging type placeholder; The step of verifying whether the target image has any anomalies based on the target prompt words, and obtaining the label verification result of the product to be packaged, includes: Based on the packaging type, the target image is analyzed to obtain various visual elements; The structured text data is compared with the various visual elements at the character level to verify the consistency between the structured text data and each visual element, and a first verification result is obtained. Using the natural language description text, determine whether the various visual elements are blurred, missing, contaminated, or have a positional offset exceeding a preset offset threshold to obtain a second verification result; perform existence identification and correctness analysis on the identifiers of the various visual elements to obtain a third verification result. The first verification result, the second verification result, and the third verification result are aggregated to obtain the anomaly probability of the product to be packaged. When the anomaly probability is greater than or equal to a preset decision threshold, a label verification result is generated indicating that the inner bottle packaging or outer box packaging of the product to be packaged is abnormal; or when the anomaly probability is less than the preset decision threshold, a label verification result is generated indicating that the inner bottle packaging or outer box packaging of the product to be packaged is not abnormal.

6. The system according to claim 5, characterized in that, The aggregation of the first verification result, the second verification result, and the third verification result to obtain the anomaly probability of the product to be packaged includes: The ratio of the number of non-matching characters in the first verification result to the total number of characters is used to obtain the first anomaly score, which is used to characterize the consistency of text content. From the second verification result, the severity of visual defects output by the pre-fine-tuned large language model for each visual element is extracted to obtain a second anomaly score used to characterize visual quality compliance. Based on the existence identification result and correctness analysis result of the identifier in the third verification result, a preset basic defect score is matched to obtain a third anomaly score used to characterize the validity of the identifier. The first abnormal score, the second abnormal score, and the third abnormal score are weighted and summed to obtain the abnormal probability of the product to be packaged.

7. The system according to any one of claims 1-6, characterized in that, Generate a pre-tuned large language model by following these steps: For each packaging production line, collect historical images of the inner bottle packaging or outer box packaging of different historical packaging products. Obtain the historical standard label data and manually annotated historical verification result labels corresponding to each historical image; Obtain a pre-trained general vision-language large model as the base model; Based on each historical image and its historical standard label data, the model training input corresponding to each historical image is generated. The model training input corresponding to each historical image and the historical verification result label corresponding to each historical image are used as training pairs and input into the basic model for iterative training. During training, the underlying parameters of the base model are frozen, and the parameters of the attention mechanism layer and adaptation layer of the base model are updated to minimize the loss value between the predicted verification result output by the base model and the historical verification result label. When the loss value converges to a minimum, a pre-fine-tuned large language model is generated.

8. The system according to claim 7, characterized in that, The step of generating model training input corresponding to each historical image based on each historical image and the historical standard label data of each historical image includes: Load a preset, dynamically instantiable, generic prompt word template; Obtain the packaging type for each historical image; Based on the packaging type of each historical image, obtain a natural language description of the preset differential verification strategy for each historical image; The packaging type of each historical image, the historical standard label data, and the natural language description of the preset differential verification strategy are filled into the corresponding placeholders of the general prompt word template to obtain the model training prompt words for each historical image. Each historical image is mapped and associated with the model training prompt words for each historical image to obtain the model training input corresponding to each historical image.

9. A product label quality inspection method for a packaging production line implemented using the system according to any one of claims 1-8, characterized in that, The method includes: The PLC control unit uses photoelectric sensors and industrial cameras to acquire target images of the inner packaging of bottles or the outer packaging of boxes, and sends them to the edge computing gateway. The edge computing gateway acquires standard label data of the production batch of the product to be packaged; based on the standard label data and the target image, and combined with a pre-tuned large language model, it generates a label verification result for the product to be packaged; when the label verification result indicates that there is an anomaly in the inner bottle packaging or outer box packaging of the product to be packaged, it generates a rejection instruction and sends it to the PLC control unit. The PLC control unit controls the rejection mechanism to perform rejection actions according to the rejection command, so as to push the product to be packaged out of the packaging production line.

Images