Automated production line equipment control method and system based on AI model code reading

Abstract

The present invention discloses an automated production line equipment control method and system based on AI model code reading, which relates to the field of equipment control technology, performs quality analysis on the acquired multi-frame image data, uses an AI model to perform a code reading subprocess based on the acquired image data sequence, compares the code value obtained by parsing the code reading subprocess with the code value actually read by the code reader, and analyzes whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface. If unqualified, an alarm signal is automatically triggered, and the PLC preset code reading fault execution process is activated, and fault data collection begins at the same time. The control system integrates industrial cameras and computing servers, realizes the rapid and accurate extraction of multiple codes, significantly improves the code reading efficiency and accuracy, solves the code reading challenges in complex scenarios, can adapt to the code reading requirements of products of different brands and models, and greatly improves the adaptability and operational flexibility of the equipment.

Description

Technical Field

[0001] The present invention relates to the field of equipment control technology, and in particular to an automated production line equipment control method and system based on AI model code reading. Background Art

[0002] With the continuous progress of society, coding technologies such as barcode, QR code, direct marking code (DM-Code) have been widely used in various industries. On the surface of many accessories, multiple codes may be printed at the same time to achieve detailed records of different information. This composite coding recording method has relatively little impact on original manufacturers because they are usually responsible for the formulation and printing of codes. However, for third-party sales manufacturers, how to automatically, intelligently, quickly and accurately extract the required code value from the numerous codes on the surface of accessories has become a challenging problem.

[0003] At present, there are two traditional barcode reading methods in the industry: manual barcode reading based on handheld barcode readers and automatic barcode reading based on industrial barcode readers.

[0004] 1. Although the manual code reading method based on handheld code readers has certain flexibility, low-cost investment, simple implementation process and flexibility of data entry, its limitations cannot be ignored;

[0005] The efficiency is low. During the manual scanning process, the code of each accessory needs to be accurately scanned one by one. Especially when processing a large amount of data, the problem of low efficiency is more prominent;

[0006] The error rate is high. Due to the instability of human operation, it is easy to miss or scan by mistake, which directly affects the accuracy and completeness of the data.

[0007] Automated management is difficult, and manual code scanning cannot be efficiently connected with modern technical facilities such as automated production lines and warehouse management systems, thus restricting the further improvement of overall operational efficiency.

[0008] 2. Although the automated barcode reading technology based on industrial barcode readers brings significant advantages to original manufacturers in terms of improving production efficiency, ensuring accuracy, improving automation level and stable operation, its limitations are also quite obvious for third-party sales manufacturers, especially those operating complex product lines with multiple brands and models:

[0009] Lack of flexibility. After installation and commissioning, the scanning range and angle of industrial barcode readers are relatively fixed, which may require additional adjustments and optimizations in specific scenarios to achieve the ideal reading effect.

[0010] Limitations of multi-code filtering solutions: When industrial barcode readers encounter multiple codes at the same time, they usually use preset regular expression rules to filter code values. However, these barcode readers support a limited number of regular expression rules, usually between 8 and 10. This filtering method works better when processing products with uniform specifications and clear code value rules. However, when faced with a large number of brands, complex categories, complicated models and inconsistent code value rules, its accuracy will be seriously affected, making it difficult to meet diverse code reading needs.

[0011] Based on the above analysis, the urgent problem that needs to be solved is how to improve the adaptability and flexibility of the equipment while ensuring the efficiency and accuracy of code reading, while reducing operating costs and overcoming the limitations of extraction and recognition in multi-code scenarios. Only in this way can we better meet the complex and changing needs faced by third-party sales manufacturers and ensure seamless and efficient integration with the automation management system. Summary of the invention

[0012] The purpose of the present invention is to provide an automated production line equipment control method and system based on AI model code reading to address the deficiencies in the background technology.

[0013] In order to achieve the above object, the present invention provides the following technical solution: an automated production line equipment control method based on AI model code reading, the control method comprising the following steps:

[0014] When the control system detects that the accessory has reached the code reading position through the sensor device, it automatically triggers the camera to take pictures and activates the code reader to start reading the code at the same time;

[0015] First, the quality of the acquired multi-frame image data is analyzed to obtain the Laplace sharpness index, contrast and noise impact index of the image, and the Laplace sharpness index, contrast and noise impact index are normalized so that the value range of the Laplace sharpness index, contrast and noise impact index is mapped to [0,1], and the Laplace sharpness index normalized value, contrast normalized value and noise impact index normalized value are obtained. The Laplace sharpness index normalized value is added to the contrast normalized value minus the noise impact index normalized value to obtain the quality coefficient. According to the comparison result of the quality coefficient and the quality threshold, it is analyzed whether the image quality is qualified, and the image data that fails the quality analysis is deleted, and it is analyzed whether the accessory image needs to be taken again;

[0016] If not, the AI ​​model is used to perform the code reading sub-process based on the acquired image data sequence, and the code value obtained by the code reading sub-process is compared with the code value actually read by the code reader to analyze whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface;

[0017] If it fails, an alarm signal will be automatically triggered, and the PLC preset code reading fault execution process will be activated, and fault data collection will begin at the same time.

[0018] In a preferred embodiment, the code reading sub-process using the AI ​​model based on the acquired image data sequence includes the following steps:

[0019] Detect and recognize the current image through the pre-trained AI model, and return the code position bounding box value;

[0020] According to the obtained code position bounding box value, the original image is cropped to obtain a cropped image containing the barcode / QR code / DM code;

[0021] Perform image optimization processing on the cropped image, including grayscale transformation, bilateral filtering and binarization operations, to obtain a preprocessed image;

[0022] Use the open source tool zxing-cpp to extract the numerical value of the barcode / QR code / DM code in the preprocessed image. If the parsing is successful, perform a quality analysis on the parsed code value based on the parsing steps and image data to determine whether the parsed code value can be used.

[0023] In a preferred embodiment, performing a quality analysis on the parsed code value based on the parsing step and the image data to determine whether the parsed code value can be used includes the following steps:

[0024] Obtain the block check failure rate and the decoder error correction ratio in the image parsing process, and obtain the image quality coefficient, and comprehensively calculate the block check failure rate, the decoder error correction ratio and the quality coefficient to obtain the decoding factor;

[0025] The decoding factor of the code value is compared with the preset defect threshold. If the decoding factor is less than the defect threshold, it is analyzed that there are no potential quality defects that affect the quality of the code value in this decoding process, and the code value obtained by analysis can be used. If the decoding factor is greater than or equal to the defect threshold, it is analyzed that there are potential quality defects that affect the quality of the code value in this decoding process, and the code value obtained by analysis cannot be used.

[0026] In a preferred embodiment, the block check failure rate, the decoder error correction ratio and the quality coefficient are comprehensively calculated to obtain the decoding factor, which is expressed as follows: , where is the decoding factor, is the block check failure rate, is the decoder error correction ratio, is the quality coefficient, , , are the adjustment coefficients of block check failure rate, decoder error correction ratio and quality coefficient, respectively, and , , Both are greater than 0.

[0027] In a preferred embodiment, deleting image data that fails the quality analysis and analyzing whether it is necessary to take the accessory image again includes the following steps:

[0028] The obtained quality coefficient is compared with the preset quality threshold. If the quality coefficient is less than the quality threshold, the image quality analysis is unqualified. If the quality coefficient is greater than or equal to the quality threshold, the image quality analysis is qualified. After deleting the image data that fails the quality analysis, the number of remaining images is obtained. If the number of remaining images is less than the preset number threshold, the analysis requires taking the accessory image again.

[0029] In a preferred embodiment, the code value obtained by parsing the code reading sub-process is compared with the code value actually read by the code reader to analyze whether the code value is qualified, including the following steps:

[0030] The code value extracted and parsed by the AI ​​model from the image cropping area is marked as AI_Code, and the code value actually read from the code reader is marked as Reader_Code;

[0031] Preprocessing the two groups of code values ​​includes removing spaces, line breaks, special characters, converting uppercase and lowercase characters, and deleting prefix or suffix interference information;

[0032] Use the string comparison method to check whether AI_Code and Reader_Code are completely consistent. The expression is: , Indicates the comparison result. If it matches, it is marked as qualified.

[0033] In a preferred embodiment, the calculation expression of the Laplace sharpness index is: , where is the Laplace sharpness index, , are the width and height of the image, respectively. Indicates the image The gray value of the pixel, Represents the image row pixel index, Represents the image Column pixel index, , It represents the second-order derivative of the image in the horizontal and vertical directions, and represents the second-order rate of change of the pixel gray value with position.

[0034] In a preferred embodiment, the calculation expression of the contrast is: , where is the contrast, is the maximum gray value of the image, is the minimum grayscale value of the image.

[0035] In a preferred embodiment, the calculation expression of the noise impact index is: , where , are the width and height of the image, respectively. Indicates the image The gray value of the pixel, represents the average gray value of the image, and .

[0036] Automated production line equipment control system based on AI model code reading, including arrival detection module, image analysis module, code value comparison module, and control module;

[0037] Arrival detection module: When the sensor device detects that the accessory has reached the code reading position, the camera is automatically triggered to take pictures and the code reader is activated to start reading the code at the same time;

[0038] Image analysis module: firstly, perform quality analysis on the acquired multi-frame image data, delete the image data that fails the quality analysis, and analyze whether it is necessary to take the accessory image again;

[0039] Code value comparison module: If not required, the AI ​​model is used to perform the code reading sub-process based on the acquired image data sequence, and the code value obtained by the code reading sub-process is compared with the code value actually read by the code reader to analyze whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface;

[0040] Control module: If it fails to meet the requirements, it will automatically trigger an alarm signal, activate the PLC preset code reading fault execution process, and start collecting fault data.

[0041] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0042] 1. When the sensor device detects that the accessory has reached the code reading position, the present invention automatically triggers the camera to shoot, and simultaneously activates the code reader to start reading the code, performs quality analysis on the acquired multi-frame image data, uses the AI ​​model to perform the code reading sub-process based on the acquired image data sequence, compares the code value obtained by the code reading sub-process with the code value actually read by the code reader, and analyzes whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface. If unqualified, an alarm signal is automatically triggered, and the PLC preset code reading fault execution process is activated, and fault data collection begins at the same time. The control system integrates industrial cameras and computing servers, realizes the rapid and accurate extraction of multiple codes, significantly improves the code reading efficiency and accuracy, solves the code reading challenges in complex scenarios, can adapt to the code reading requirements of products of different brands and models, and greatly improves the adaptability and operational flexibility of the equipment;

[0043] 2. The present invention obtains the quality coefficient by adding the Laplace sharpness index normalized value to the contrast normalized value minus the noise influence index normalized value. The smaller the quality coefficient of the image, the worse the quality of the image. The obtained quality coefficient is compared with a preset quality threshold. If the quality coefficient is less than the quality threshold, the image quality is analyzed to be unqualified. If the quality coefficient is greater than or equal to the quality threshold, the image quality is analyzed to be qualified. Before processing the image data, the image quality is analyzed, so that the image with unqualified quality can be deleted in advance, effectively avoiding the use of the image with unqualified quality, which is beneficial to improving the accuracy of code value analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0045] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Example 1: Please refer to Figure 1As shown, the automated production line equipment control method based on AI model code reading described in this embodiment includes the following steps:

[0048] When the control system detects that the accessory has reached the code reading position through the sensor device, it automatically triggers the camera to shoot and activates the code reader to start reading the code, including reading barcodes, QR codes or DM codes. First, the quality analysis of the acquired multi-frame image data is performed, and the image data that fails the quality analysis is deleted. It is analyzed whether the accessory image needs to be taken again. If not, the AI ​​model is used to perform the code reading sub-process based on the acquired image data sequence, and the code value obtained by the code reading sub-process is compared with the code value actually read by the code reader to analyze whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface. If unqualified, an alarm signal is automatically triggered, and the PLC preset code reading fault execution process is activated, and fault data collection begins at the same time. The system code reading interface displayed on the front end is mainly responsible for displaying the feedback and statistics of the system code reading information. When the code reading information sent by the server is received, it is first determined whether it is a fault information. If it is normal code reading information, the code value is directly displayed and the code reading list is refreshed; otherwise, the fault information will be prompted to notify the on-site operator to execute the corresponding fault handling and recovery process.

[0049] This application uses a sensor device to detect when the accessory reaches the code reading position, automatically triggers the camera to shoot, and activates the code reader to start reading the code at the same time, performs quality analysis on the acquired multi-frame image data, uses an AI model to perform a code reading sub-process based on the acquired image data sequence, compares the code value obtained by the code reading sub-process with the code value actually read by the code reader, and analyzes whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface. If unqualified, an alarm signal is automatically triggered, and the PLC preset code reading fault execution process is activated, and fault data collection begins at the same time. The control system integrates industrial cameras and computing servers, realizes the rapid and accurate extraction of multiple codes, significantly improves the code reading efficiency and accuracy, solves the code reading challenges in complex scenarios, can adapt to the code reading needs of products of different brands and models, and greatly improves the adaptability and operational flexibility of the equipment.

[0050] The key technical terms involved in the present invention are described in Table 1:

[0051]

[0052] 1. The image acquisition process is highly automated on the automated production line. When the sensor detects that the accessory has reached the specified position, the system automatically triggers the camera to take a picture and simultaneously activates the industrial code reader to start reading the barcode, QR code or DM code. This operation will instantly start the code reading system and put it into working state, ready to process and analyze the collected image data.

[0053] 2. After successfully capturing the image, the code reading system will sequentially trigger the subsequent four sub-processes based on the acquired image data to complete the entire code reading operation.

[0054] (1) Code position detection: This step mainly relies on the artificial intelligence image detection model. The model detects and recognizes the current image based on the training results and returns the most likely code position bounding box (bbox) value. The bbox value is crucial in the entire code reading process because it directly determines the success rate and accuracy of code reading.

[0055] (2) Code area cropping. In order to improve the speed and accuracy of code reading and effectively avoid the problem of multiple code confusion, this process will accurately crop the original image based on the bbox value obtained in the previous step. After cropping, we get an image that only contains the required barcode / QR code / DM code, which is then returned for use in subsequent steps. This process ensures the pertinence and efficiency of the code reading operation.

[0056] (3) Code area optimization processing: In order to further improve the success rate of code analysis, this step will perform a series of image optimization processing on the image obtained in the previous step. This includes grayscale transformation, bilateral filtering, and binarization operations, aiming to obtain a clearer black and white outline image with high contrast. These optimization processes help enhance the features of the barcode or QR code, thereby improving the accuracy and efficiency of subsequent analysis.

[0057] (4) Code parsing: This step is the core of the entire process. The key lies in the in-depth analysis of the image that has been optimized in the previous stage. By using the open source tool zxing-cpp, the system can accurately extract the value of the barcode / QR code / DM code in the image. Once the code value is successfully parsed, the system will automatically trigger the subsequent quality control process to ensure that the data is accurate and meets the quality standard requirements.

[0058] 3. The quality control link is a crucial part of the entire process. In this link, the system will accurately compare the code value obtained by parsing in the above steps with the code value actually read by the industrial code reader (which may include multiple). At the same time, the system will also conduct a comprehensive check on the total score of this image detection to ensure that it meets the preset qualification standards. Once the code value is confirmed to be correct and the total score meets the standard, the system will immediately feedback the qualified code result to the front-end code reading interface in real time for the operator to refer to. On the contrary, if a difference in the code value is found during the comparison process or the total score does not meet the qualification standard, the system will automatically trigger the fault handling process to timely troubleshoot the problem and take effective measures to repair it. This link is designed to ensure the accuracy and reliability of the code reading results, providing strong support for subsequent production and quality control.

[0059] 4. The fault handling link is an indispensable part of the entire process. It is mainly responsible for handling faults that occur during the detection process and collecting images with unqualified scores as the corpus basis for subsequent incremental model training. When the fault handling process is triggered, the system will immediately activate the fault execution process preset by the PLC and start collecting fault data. During this process, the system will also issue an alarm signal to remind the operator to pay attention and intervene in time. Subsequently, the operator will manually handle the fault according to the alarm information and confirm the cause of the fault. Once the fault is properly resolved and confirmed, the system will resume production operation. This link is designed to ensure the stability and reliability of the system and provide strong guarantees for subsequent production and quality control.

[0060] The system code reading interface displayed on the front end is mainly responsible for displaying the feedback and statistics of the system code reading information. When receiving the code reading information sent by the server, first determine whether it is a fault information. If it is normal code reading information, the code value is directly displayed and the code reading list is refreshed; otherwise, the fault information will be prompted to notify the on-site operator to perform the corresponding fault handling and recovery process.

[0061] The production line integrated automation part belongs to the infrastructure layer, which aims to integrate a series of sophisticated industrial automation equipment with a flexible and controllable image acquisition system. Specifically, the system integrates the following key devices, each of which plays an indispensable role:

[0062] Production line equipment: Traditional tooling machines are used as the production line body, responsible for accurately transporting and positioning workpieces to ensure the smooth progress of the production process.

[0063] PLC (Programmable Logic Controller): As the backbone of the automation system, PLC is responsible for performing complex logical operations and controlling various operations of the production line.

[0064] Frequency converter: adjusts the motor speed to achieve precise control of equipment speed to meet different production needs.

[0065] Sensors: carefully deployed at key locations, responsible for detecting whether the workpiece has accurately arrived at the image acquisition area. They play a vital role, not only triggering the industrial camera to take precise photos to ensure real-time capture of image data, but also activating the industrial barcode reader to perform efficient barcode reading operations, thereby ensuring that the information flow and logistics of the entire production line are synchronized and correct.

[0066] Automated bracket: Provides stable camera support and can adjust shooting angle, position and height according to production needs.

[0067] Industrial camera: It uses a high-performance 10G area array camera with a precisely controlled lens. The key to high-definition image acquisition is to capture workpiece details through precise control, providing a high-quality data source for subsequent image processing and analysis.

[0068] Industrial code reader: Use a mature 20-megapixel code reader with a precisely controlled lens to capture all codes on the workpiece surface, providing basic data for subsequent inspections and improving code reading accuracy.

[0069] The integration of these devices not only improves the automation level of the production line, but also lays a solid foundation for achieving a high-efficiency, low-error production process.

[0070] This invention patent proposes an automated production line code reading device and system based on artificial intelligence, which has the following significant advantages over the existing technology:

[0071] Efficiency and accuracy: This invention integrates high-precision industrial cameras and high-performance computing servers to achieve fast and accurate extraction of multiple codes, significantly improving the efficiency and accuracy of code reading and solving the challenges of code reading in complex scenarios.

[0072] Equipment adaptability and flexibility: Compared with traditional industrial code readers, the present invention adopts a flexible automated bracket and an adjustable industrial camera, which can adapt to the code reading requirements of accessories of different brands and models, greatly improving the adaptability and operational flexibility of the equipment.

[0073] Reduce operating costs: The automated barcode reading system reduces the need for manual operations, reduces rework and scrap rates caused by human errors, and significantly reduces operating costs.

[0074] Powerful multi-code processing capability: The present invention adopts advanced AI image recognition technology, which can effectively handle the situation where multiple codes exist at the same time, overcomes the limitations of traditional code readers in multi-code scenarios, and improves the accuracy and efficiency of code reading.

[0075] Integration of comprehensive management system: The present invention provides a comprehensive management system covering multiple modules such as code reading information display, model management and system settings, realizing the full life cycle management of the code reading process, facilitating the rapid training and optimization of the model, as well as the management and control of automated equipment.

[0076] Fault feedback and quality control: The present invention has an intelligent fault feedback mechanism, which can timely detect and handle faults in the code reading process, ensuring the stable operation of the production line. At the same time, the quality control link ensures the accuracy and reliability of the code reading results.

[0077] User-friendliness: The front-end display system's code reading interface is intuitive and clear, making it easy for operators to monitor code reading status and handle fault information, improving user experience and work efficiency.

[0078] Model iteration and optimization: The model management module supports rapid data annotation, data set management, and model training and services, enabling image recognition models to be continuously iterated and optimized to adapt to changing production needs.

[0079] In summary, the patent of this invention has obvious advantages in improving code reading efficiency, enhancing equipment adaptability, reducing operating costs, optimizing management processes, etc. It provides efficient and intelligent code reading solutions for third-party sales manufacturers, ensuring seamless and efficient integration with automated management systems.

[0080] Embodiment 2:

[0081] When the control system detects that the accessory has reached the code reading position through the sensor device, it automatically triggers the camera to take pictures and activates the code reader to start reading the code, including reading barcodes, QR codes or DM codes, including the following steps:

[0082] Equipment operation status monitoring: Sensors (such as photoelectric sensors, proximity switches or laser detection devices) operate in real time to detect whether accessories enter the code reading area.

[0083] Detection method:

[0084] Photoelectric sensor: Confirms that the accessory has reached the code reading position by blocking the light beam.

[0085] Proximity sensor: triggers a signal when a metal or non-metal accessory is detected within a certain range.

[0086] Other sensors: Such as vision sensors, which can identify the shape, size or color of accessories.

[0087] Environmental adaptability: The sensor needs to adapt to the light, vibration and other interference conditions on site to ensure reliable detection.

[0088] Signal generation: When the sensor detects that the accessory has reached a predetermined position, it immediately sends a trigger signal to the controller (PLC or industrial computer).

[0089] Signal transmission method: through I / O interface (such as digital signal output). Use industrial communication protocols (such as Modbus, Profinet or EtherCAT) for data transmission.

[0090] Signal confirmation: After receiving the sensor signal, the controller quickly performs logical judgment to confirm the validity of the signal: determine whether it is a real detection signal (excluding interference signals). Confirm whether the signal time meets the system preset time window (to avoid repeated triggering or wrong triggering).

[0091] Start command sending: The controller sends a shooting command to the camera to start the shooting process.

[0092] Multi-frame capture: The camera may capture multiple frames of images continuously to improve the accuracy of data collection.

[0093] Shooting parameter control: According to the accessory position data provided by the sensor signal, adjust the shooting angle, light source brightness and other parameters to ensure a clear image.

[0094] Start the code reader: The controller sends a start instruction to the code reader synchronously and enters the code reading state.

[0095] Read code value: The code reader starts scanning the barcode, QR code or DM code on the accessory and tries to directly resolve the code value.

[0096] Synchronous with shooting data: The image data collected by the camera and the data analyzed by the code reader will be transmitted to the control system at the same time.

[0097] The working sequence and rhythm of the two are coordinated by the control system to avoid data loss or processing conflicts.

[0098] Performing quality analysis on the acquired multiple frames of image data, deleting image data that fails the quality analysis, and analyzing whether it is necessary to take accessory images again, includes the following steps:

[0099] The Laplace sharpness index, contrast and noise influence index of the image are obtained, and the Laplace sharpness index, contrast and noise influence index are normalized so that the value ranges of the Laplace sharpness index, contrast and noise influence index are mapped to [0,1], and the Laplace sharpness index normalized value, contrast normalized value and noise influence index normalized value are obtained. The quality coefficient is obtained by adding the Laplace sharpness index normalized value to the contrast normalized value and subtracting the noise influence index normalized value;

[0100] The smaller the quality coefficient of the image, the worse the quality of the image. The obtained quality coefficient is compared with the preset quality threshold. If the quality coefficient is less than the quality threshold, the image quality analysis is unqualified. If the quality coefficient is greater than or equal to the quality threshold, the image quality analysis is qualified. After deleting the image data that fails the quality analysis, the number of remaining images is obtained. If the number of remaining images is less than the preset number threshold, the analysis needs to take the accessory image again.

[0101] The present application obtains the quality coefficient by adding the Laplace sharpness index normalized value to the contrast normalized value minus the noise impact index normalized value. The smaller the image quality coefficient, the worse the image quality. The obtained quality coefficient is compared with a preset quality threshold. If the quality coefficient is less than the quality threshold, the image quality is analyzed to be unqualified. If the quality coefficient is greater than or equal to the quality threshold, the image quality is analyzed to be qualified. Before processing the image data, the image quality is analyzed, so that images with unqualified quality can be deleted in advance, effectively avoiding the use of images with unqualified quality, which is beneficial to improving the accuracy of code value analysis.

[0102] The calculation expression of Laplace sharpness index is: , where is the Laplace sharpness index, , are the width and height of the image (in pixels), Indicates the image The grayscale value of the pixel (usually in the range of 0 to 255 for grayscale images), Represents the image row pixel index, Represents the image Column pixel index, , It represents the second-order derivative of the image in the horizontal and vertical directions, and represents the second-order rate of change of the pixel gray value with position. The second-order derivative is obtained by approximate calculation of the discrete Laplace operator, which belongs to the prior art and will not be repeated in this application.

[0103] The Laplace sharpness index is the second-order rate of change of the grayscale value of pixels in an image. When the image is clear, the grayscale value of edges and details changes significantly, resulting in a large second-order derivative and thus a large Laplace sharpness index. In a blurred image, the grayscale value changes slowly, the gradient difference between pixels is small, and the second-order derivative is close to zero, resulting in a low Laplace sharpness index.

[0104] Image quality relationship:

[0105] High Laplace sharpness index → ​​high clarity → image is suitable for code reading.

[0106] Low Laplace sharpness index → ​​low clarity → image may lead to recognition failure.

[0107] The calculation expression of contrast is: , where is the contrast, is the maximum gray value of the image, is the minimum grayscale value of the image.

[0108] High contrast: The grayscale difference between the object and the background in the image is obvious, and the barcode / QR code is clear and easy to read.

[0109] Low contrast: The grayscale difference between the object and the background in the image is small, and the barcode / QR code may not be clear.

[0110] Contrast refers to the difference in grayscale values ​​between bright and dark areas in an image. When the contrast is high, the boundaries of the barcode / QR code in the image are clearer, and the recognition algorithm can more accurately extract the barcode or QR code area.

[0111] Low contrast may be caused by insufficient light (dim) or overexposure (too bright), resulting in a decrease in the grayscale difference between the barcode / QR code and the background, and unclear image details.

[0112] Image quality relationship:

[0113] High contrast → barcode / QR code is clear and background is distinguishable → image is suitable for code reading.

[0114] Low contrast → blurry barcode / QR code boundaries, background and barcode confused → poor image quality.

[0115] The calculation expression of noise impact index is: , where , are the width and height of the image (in pixels), Indicates the image The grayscale value of the pixel (usually in the range of 0 to 255 for grayscale images), represents the average gray value of the image, and .

[0116] Low noise level: The image is smooth and the details are clear.

[0117] High noise level: The image contains a large number of noise points, and the interference information is significant.

[0118] The noise level (standard deviation of image grayscale values) reflects the degree to which the pixel grayscale values ​​in the image deviate from the average grayscale. When the noise level is high, random noise points in the image (such as Gaussian noise and salt and pepper noise) will cover the key feature areas of the barcode or QR code, affecting the recognition accuracy.

[0119] When the noise level is low, the image is smoother, the noise interferes less with the barcode / QR code boundary, and it is easier to recognize.

[0120] Image quality relationship:

[0121] Low noise level → clean image → prominent barcode / QR code details → high image quality.

[0122] High noise level → image is cluttered → barcode / QR code details are obscured → poor image quality.

[0123] If not required, the AI ​​model is used to perform the code reading sub-process based on the acquired image data sequence, including the following steps:

[0124] Detect and recognize the current image through the pre-trained AI model, and return the code position bounding box value;

[0125] According to the obtained code position bounding box value, the original image is cropped to obtain a cropped image containing the barcode / QR code / DM code;

[0126] Perform image optimization processing on the cropped image, including grayscale transformation, bilateral filtering and binarization operations, to obtain a preprocessed image;

[0127] Use the open source tool zxing-cpp to extract the numerical value of the barcode / QR code / DM code in the preprocessed image. If the parsing is successful, perform a quality analysis on the parsed code value based on the parsing steps and image data to determine whether the parsed code value can be used.

[0128] (1) Code position detection: This step mainly relies on the artificial intelligence image detection model. The model detects and recognizes the current image based on the training results and returns the most likely code position bounding box (bbox) value. The bbox value is crucial in the entire code reading process because it directly determines the success rate and accuracy of code reading.

[0129] (2) Code area cropping. In order to improve the speed and accuracy of code reading and effectively avoid the problem of multiple code confusion, this process will accurately crop the original image based on the bbox value obtained in the previous step. After cropping, we get an image that only contains the required barcode / QR code / DM code, which is then returned for use in subsequent steps. This process ensures the pertinence and efficiency of the code reading operation.

[0130] (3) Code area optimization processing: In order to further improve the success rate of code analysis, this step will perform a series of image optimization processing on the image obtained in the previous step. This includes grayscale transformation, bilateral filtering, and binarization operations, aiming to obtain a clearer black and white outline image with high contrast. These optimization processes help enhance the features of the barcode or QR code, thereby improving the accuracy and efficiency of subsequent analysis.

[0131] (4) Code parsing: This step is the core of the entire process. The key lies in the in-depth analysis of the image that has been optimized in the previous stage. By using the open source tool zxing-cpp, the system can accurately extract the value of the barcode / QR code / DM code in the image. Once the code value is successfully parsed, the system will automatically trigger the subsequent quality control process to ensure that the data is accurate and meets the quality standard requirements.

[0132] Pre-training of AI models includes the following steps:

[0133] Collect a large amount of image data including barcodes, QR codes, DM codes, etc.

[0134] Diversity: Includes images of different types, sizes, resolutions, and lighting conditions.

[0135] Background complexity: covers simple backgrounds to complex scenes (such as industrial production lines).

[0136] Use the annotation tool to annotate the target area in the image and obtain the bounding box coordinates and code type label.

[0137] Label types: barcode, QR code, DataMatrix code, etc.

[0138] Input augmented images and annotation data, including bounding boxes and category labels.

[0139] The feature map is extracted using a pre-trained VGG model, which is a prior art step.

[0140] With each feature point as the center, candidate boxes of various proportions and sizes are generated, and the candidate boxes are compared with the real annotated target boxes to calculate the IoU (Intersection-over-Union). According to the IoU value, they are divided into positive samples (high matching degree with the real box) and negative samples (low matching degree with the real box).

[0141] Use the candidate box output by RPN to perform fixed-size feature extraction on the candidate area of ​​the feature map, adjust the position of the candidate box to make it closer to the real target box, and classify each candidate box as a barcode, QR code, DM code or background. In each training round, gradually optimize the AI ​​model parameters to reduce the total loss.

[0142] AI model output:

[0143] Bounding box coordinates: The location of the predicted code.

[0144] Category label: barcode, QR code or DM code.

[0145] Confidence score: How reliable the bounding box and classification results are.

[0146] The cropped image is subjected to image optimization processing, including grayscale transformation, bilateral filtering, and binarization operations, to obtain a preprocessed image, including the following steps:

[0147] Convert the cropped image from color space (such as RGB) to grayscale space, remove color information, and retain intensity information. Convert the cropped image from color space (such as RGB) to grayscale space, remove color information, and retain intensity information. Each pixel value is between [0,255]. Bilateral filtering is an edge-preserving denoising method that can retain edge information while smoothing the image. Binarization divides the pixels in the grayscale image into two categories: foreground (code information) and background.

[0148] Use the open source tool zxing-cpp to extract the value of the barcode / QR code / DM code in the preprocessed image, including the following steps:

[0149] Make sure zxing-cpp is installed. You can get it from Get the source code for compilation. If you use Python binding, you can install the relevant toolkit through pipinstallpyzxing and other methods.

[0150] Use the image processing library to load the preprocessed binary image. The relevant code is as follows:

[0151] from-PIL-import-Image;

[0152] image=Image.open("preprocessed_image.png").convert("L");

[0153] In Python, the PIL image needs to be converted into a file path or byte stream format for use by zxing.

[0154] Initialize the decoder. The relevant code is as follows:

[0155] reader = Bar-Code-Reader();

[0156] Decode the image, the relevant code is as follows:

[0157] result=reader.decode("preprocessed_image.png");

[0158] ifresult:

[0159] forcodeinresult:

[0160] print("DecodedText:",code["parsed"]);

[0161] print("Format:",code["format"]);

[0162] else:

[0163] print("No-barcode-detected.");

[0164] Check that the return value contains valid decoded content:

[0165] Decoded-Text or parsed.

[0166] Barcode format (such as QR_CODE, DATA_MATRIX, CODE_128, etc.).

[0167] Position data (such as optional bounding box information).

[0168] Save the decoding results to the log or pass them to the front-end system:

[0169] If the decoding is successful, record the code value and format.

[0170] If decoding fails, an error signal is returned, prompting you to reshoot or check the image quality.

[0171] Successful decoding: Extract the barcode / QR code / DM code value and return text information (such as barcode number, URL or other content).

[0172] Failure processing: triggers the reshoot process or marks it as undecodeable for further manual or automatic processing.

[0173] Performing a quality analysis on the parsed code value based on the parsing steps and the image data to determine whether the parsed code value can be used includes the following steps:

[0174] Obtain the block check failure rate and decoder error correction ratio during the image parsing process, and obtain the image quality coefficient. The block check failure rate, decoder error correction ratio and quality coefficient are calculated comprehensively to obtain the decoding factor, which is expressed as: , where is the decoding factor, is the block check failure rate, is the decoder error correction ratio, is the quality coefficient, , , are the adjustment coefficients of block check failure rate, decoder error correction ratio and quality coefficient, respectively, and , , All are greater than 0;

[0175] The larger the value of the decoding factor, the greater the potential quality defect of the code value (that is, the lower the confidence of the code value). Compare the decoding factor of the code value with the preset defect threshold. If the decoding factor is less than the defect threshold, the analysis shows that there are no potential quality defects that affect the quality of the code value in this decoding process, and the code value obtained by the analysis can be used. If the decoding factor is greater than or equal to the defect threshold, the analysis shows that there are potential quality defects that affect the quality of the code value in this decoding process, and the code value obtained by the analysis cannot be used.

[0176] The calculation logic of the block verification failure rate is as follows: obtain the number of blocks that failed verification and the total number of all blocks in the barcode, and divide the number of blocks that failed verification by the total number of all blocks in the barcode to obtain the block verification failure rate.

[0177] Lower failure rate: indicates higher integrity of the barcode area and less data corruption.

[0178] High failure rate: May indicate poor barcode printing, physical damage, or poor photography quality.

[0179] The calculation logic of the decoder error correction ratio is: obtain the number of erroneous data blocks successfully recovered by the error correction mechanism, obtain the total number of erroneous data blocks detected by the decoder, and divide the number of erroneous data blocks successfully recovered by the error correction mechanism by the total number of erroneous data blocks detected by the decoder to obtain the decoder error correction ratio. A larger decoder error correction ratio indicates that there is damage in the barcode area.

[0180] Compare the code value obtained by the code reading sub-process with the code value actually read by the code reader to analyze whether the code value is qualified, including the following steps:

[0181] The code value extracted and parsed from the image cropped area using the AI ​​model is marked as AI_Code. This value may come from the recognition result of a QR code, barcode, or DM code. The code value actually read by the code reader is marked as Reader_Code.

[0182] Preprocess the two sets of code values ​​to ensure consistent format:

[0183] Remove spaces, line breaks, and special characters.

[0184] Convert case (e.g., all lowercase or all uppercase).

[0185] Remove possible prefix or suffix interference information (such as identifiers, control characters).

[0186] Check whether the two sets of code values ​​meet the predetermined format requirements, for example:

[0187] Number: whether it is a pure numeric sequence, check the length.

[0188] String: whether it contains the specified character set.

[0189] Checksum: such as whether the barcode contains a valid check digit.

[0190] Use the string comparison method to check whether AI_Code and Reader_Code are completely consistent. The expression is: , Indicates the comparison result. If it matches, it is marked as qualified. According to the comparison result, the code value is classified:

[0191] Completely consistent: indicates that the code value is qualified and can proceed to the next step.

[0192] Partial match: indicates that there may be minor problems and manual review is required.

[0193] Completely inconsistent: the code value is considered unqualified and an alarm signal is triggered.

[0194] If qualified, the qualified code value will be fed back to the front-end code reading interface. If unqualified, an alarm signal will be automatically triggered, and the PLC preset code reading fault execution process will be activated, and fault data will be collected at the same time, including the following steps:

[0195] Feedback the qualified code value to the front-end code reading interface:

[0196] When Match is True, that is, AI_Code and Reader_Code are completely consistent, the code value is considered qualified.

[0197] The system feeds back the qualified code value to the front-end interface and updates the display area.

[0198] Update code value display: Display the feedback qualified code value in the code reading box on the user interface.

[0199] Refresh code reading list: The system will update the current code reading list and add the latest valid code value.

[0200] Display status information: The interface will display the status information of the current operation, such as "Code value is valid", and may also display relevant statistical data (such as the total number of successful reads, code reading success rate, etc.).

[0201] The system stores the qualified code value and related information (such as timestamp, production batch, equipment number, etc.) in the database. Update the tracking records of the production process to ensure data integrity and traceability.

[0202] When Match is False or the code value is determined to be unqualified through other checking mechanisms (such as fuzzy matching failure, checksum mismatch, etc.), the system triggers an alarm signal.

[0203] Alert Type:

[0204] Audible alarm: Sounds a warning sound through a siren or speaker.

[0205] Visual Alert: Alerts the operator via a flashing red light or other visual signal.

[0206] System log recording: Record alarm events and related error information in the system log to facilitate subsequent troubleshooting and analysis.

[0207] The PLC (Programmable Logic Controller) automatically activates the code reading fault process according to the preset fault handling logic.

[0208] Pause the production line: Pause the operation of the current production line to prevent unqualified products from continuing to flow into the subsequent process.

[0209] Retake / Reread: Automatically adjust the camera or barcode reader position or brightness and retake the picture or reread the barcode.

[0210] Manual inspection notification: If automatic processing fails, the PLC may initiate a manual intervention process to notify the operator to perform manual processing.

[0211] The system starts recording and collecting relevant fault data to facilitate subsequent analysis and troubleshooting.

[0212] Fault type: such as image quality problems, hardware failures (such as damaged barcode readers), lighting problems, etc.

[0213] Fault occurrence time: record the time point when each fault occurs.

[0214] Image data: Save relevant failed image data (such as cropped images, original images) in the log for subsequent analysis.

[0215] Production batch: Record the specific production batch or equipment number where the failure occurred to ensure that the problem can be traced.

[0216] Operator records: record the actions and feedback taken by on-site operators when handling faults.

[0217] Embodiment 3: The automated production line equipment control system based on AI model code reading described in this embodiment includes an arrival detection module, an image analysis module, a code value comparison module, and a control module;

[0218] Arrival detection module: When the sensor device detects that the accessory has reached the code reading position, the camera is automatically triggered to take pictures, and the code reader is activated to start reading the code at the same time. The pictures taken are sent to the picture analysis module, and the code reading results are sent to the code value comparison module;

[0219] Image analysis module: firstly, perform quality analysis on the acquired multi-frame image data, delete the image data that fails the quality analysis, and analyze whether it is necessary to take the accessory image again. The analysis results are sent to the code value comparison module;

[0220] Code value comparison module: If not required, the AI ​​model is used to perform the code reading sub-process based on the acquired image data sequence, and the code value obtained by the code reading sub-process is compared with the code value actually read by the code reader to analyze whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface, and the code value analysis result is sent to the control module;

[0221] Control module: If it fails to meet the requirements, it will automatically trigger an alarm signal, activate the PLC preset code reading fault execution process, and start collecting fault data.

[0222] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0223] It should be understood that the term "and / or" in this article only describes the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0224] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0225] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0226] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. The automated production line equipment control method based on AI model code reading is characterized by: The control method comprises the following steps: When the control system detects that the accessory has reached the code reading position through the sensor device, it automatically triggers the camera to take pictures and activates the code reader to start reading the code at the same time; First, the quality of the acquired multi-frame image data is analyzed to obtain the Laplace sharpness index, contrast and noise impact index of the image, and the Laplace sharpness index, contrast and noise impact index are normalized so that the value range of the Laplace sharpness index, contrast and noise impact index is mapped to [0,1], and the Laplace sharpness index normalized value, contrast normalized value and noise impact index normalized value are obtained. The Laplace sharpness index normalized value is added to the contrast normalized value minus the noise impact index normalized value to obtain the quality coefficient. According to the comparison result of the quality coefficient and the quality threshold, it is analyzed whether the image quality is qualified, and the image data that fails the quality analysis is deleted, and it is analyzed whether the accessory image needs to be taken again; If not, the AI ​​model is used to perform the code reading sub-process based on the acquired image data sequence, and the code value obtained by the code reading sub-process is compared with the code value actually read by the code reader to analyze whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface; If it fails, an alarm signal will be automatically triggered, and the PLC preset code reading fault execution process will be activated, and fault data collection will begin at the same time; Performing a quality analysis on the parsed code value based on the parsing steps and the image data to determine whether the parsed code value can be used includes the following steps: Obtain the block check failure rate and decoder error correction ratio during the image parsing process, and obtain the image quality coefficient. The block check failure rate, decoder error correction ratio and quality coefficient are calculated comprehensively to obtain the decoding factor, which is expressed as: , where is the decoding factor, is the block check failure rate, is the decoder error correction ratio, is the quality coefficient, , , are the adjustment coefficients of block check failure rate, decoder error correction ratio and quality coefficient, respectively, and , , All are greater than 0; The decoding factor of the code value is compared with the preset defect threshold to determine whether the parsed code value can be used.

2. The automated production line equipment control method based on AI model code reading according to claim 1 is characterized in that: Deleting image data that fails the quality analysis and analyzing whether it is necessary to take the accessory image again includes the following steps: The obtained quality coefficient is compared with the preset quality threshold. If the quality coefficient is less than the quality threshold, the image quality analysis is unqualified. If the quality coefficient is greater than or equal to the quality threshold, the image quality analysis is qualified. After deleting the image data that fails the quality analysis, the number of remaining images is obtained. If the number of remaining images is less than the preset number threshold, the analysis requires taking the accessory image again.

3. The automated production line equipment control method based on AI model code reading according to claim 2 is characterized in that: Compare the code value obtained by the code reading sub-process with the code value actually read by the code reader to analyze whether the code value is qualified, including the following steps: The code value extracted and parsed by the AI ​​model from the image cropping area is marked as AI_Code, and the code value actually read from the code reader is marked as Reader_Code; Preprocessing the two groups of code values ​​includes removing spaces, line breaks, special characters, converting uppercase and lowercase characters, and deleting prefix or suffix interference information; Use the string comparison method to check whether AI_Code and Reader_Code are completely consistent. The expression is: , Indicates the comparison result. If it matches, it is marked as qualified.

4. The automated production line equipment control method based on AI model code reading according to claim 3 is characterized in that: The calculation expression of the Laplace sharpness index is: , where is the Laplace sharpness index, , are the width and height of the image, respectively. Indicates the image The gray value of the pixel, Represents the image row pixel index, Represents the image Column pixel index, , It represents the second-order derivative of the image in the horizontal and vertical directions, and represents the second-order rate of change of the pixel gray value with position.

5. The automated production line equipment control method based on AI model code reading according to claim 4 is characterized in that: The calculation expression of the contrast is: , where is the contrast, is the maximum gray value of the image, is the minimum grayscale value of the image.

6. The automated production line equipment control method based on AI model code reading according to claim 5 is characterized in that: The calculation expression of the noise impact index is: , where , are the width and height of the image, respectively. Indicates the image The gray value of the pixel, represents the average gray value of the image, and .

7. An automated production line equipment control system based on AI model code reading, used to implement the control method described in any one of claims 1 to 6, characterized in that: It includes arrival detection module, image analysis module, code value comparison module and control module; Arrival detection module: When the sensor device detects that the accessory has reached the code reading position, the camera is automatically triggered to take pictures and the code reader is activated to start reading the code at the same time; Image analysis module: firstly, perform quality analysis on the acquired multi-frame image data, delete the image data that fails the quality analysis, and analyze whether it is necessary to take the accessory image again; Code value comparison module: If not required, the AI ​​model is used to perform the code reading sub-process based on the acquired image data sequence, and the code value obtained by the code reading sub-process is compared with the code value actually read by the code reader to analyze whether the code value is qualified. If qualified, the qualified code value is fed back to the front-end code reading interface; Control module: If it fails to meet the requirements, it will automatically trigger an alarm signal, activate the PLC preset code reading fault execution process, and start collecting fault data.

Images