A filling status monitoring system and method

Abstract

The present invention discloses a filling status monitoring system and method, and relates to the technical field of filling monitoring. The system includes a machine vision detection device for detecting the filling status of a packaging bottle, a machine vision detection module for collecting video images and constructing a preprocessed image set, an image data analysis module for obtaining the image to be processed for reprocessing and judging whether the packaging bottle is leaking through image analysis, and outputting a first data analysis result; in the machine vision detection module, the video stream obtained after the detection is also converted into a video image and collected in an image data set; video images are also collected from the image data set and preprocessed to generate a preprocessed image set. The present invention makes full use of the advantages of machine vision and image processing and uses them to detect the filling status of the packaging bottle, thereby improving the accuracy and comprehensiveness of the detection; and through feature classification and similarity judgment, the video image is made more complete, ensuring the reliability of the image analysis.

Description

Technical Field

[0001] The invention belongs to the technical field of filling monitoring, and in particular relates to a filling state monitoring system and method. Background Art

[0002] In industrial production, the sealing of packaging bottles is crucial to product quality. Ensuring that the packaging bottles are well sealed and leak-free can effectively protect the products from contamination and damage. Therefore, in the production of packaging bottles, how to effectively prevent the packaging bottles from leaking is particularly important. In the prior art, when leak detection is performed on packaging bottles, a detection method is usually used to detect the air pressure of empty bottles, and the pressure is monitored to determine whether the packaging bottle is leaking; however, this method of detecting air pressure cannot accurately determine the location of the leakage, nor can it determine whether it is caused by cracks in the bottle body or poor sealing at the bottle mouth. In addition, for liquids, a single air pressure detection cannot accurately determine the state of the packaging bottle. Therefore, a more comprehensive monitoring method / means is urgently needed to effectively detect the leakage of packaging bottles. Summary of the invention

[0003] The object of the present invention is to provide a filling status monitoring system and method, and the object of the present invention can be achieved through the following technical solutions:

[0004] In a first aspect, an embodiment of the present application provides a filling status monitoring system, including a machine vision detection module, an image data analysis module, a filling air pressure detection module, an air pressure data analysis module, a leakage model building module, an abnormal prompt processing module and a status record feedback module;

[0005] The machine vision detection module uses a machine vision detection device to detect the filling status of the packaging bottle, collect video images and construct a pre-processed image set;

[0006] The image data analysis module is used to obtain the image to be processed, perform reprocessing, and determine whether the packaging bottle is leaking through image analysis, and output a first data analysis result;

[0007] The filling air pressure detection module establishes an air pressure judgment standard, uses an air pressure detection device to detect the filling status of the packaging bottle, obtains air pressure detection data and constructs an air pressure data set;

[0008] The air pressure data analysis module is used to obtain the data to be processed, perform data processing, and determine whether the packaging bottle is leaking through data analysis, and output a second data analysis result;

[0009] The leakage model building module builds a filling leakage model based on the data analysis results and determines the location where the leakage occurs on the packaging bottle;

[0010] The abnormality prompt processing module is used to mark the leaking packaging bottle as an abnormal product and perform abnormal prompt and abnormal processing;

[0011] The status recording and feedback module is used to record the test results and abnormal conditions and conduct filling evaluation and feedback, and then adjust and control the production process;

[0012] Wherein, the machine vision detection device and the air pressure detection device perform detection and collect data at the same time, thereby generating the same timestamp;

[0013] The image data analysis module includes an image analysis unit, which is used to perform image analysis on the reprocessed image, determine whether the packaging bottle leaks and output a first data analysis result, which specifically includes the following contents:

[0014] Region positioning subunit: obtain the best matching point in the grayscale histogram as the image positioning point and obtain the image offset by calculating the coordinate difference between two consecutive image positioning points, so as to locate according to the image positioning point and move the same distance as the image offset to the feature area;

[0015] Feature extraction subunit: performs feature extraction in the feature area; the feature area includes the bottle mouth area, the bottle body area, the bottle bottom area and the filling liquid area;

[0016] Feature analysis subunit: performs feature analysis on the extracted features and makes a decision on whether the packaging bottle is leaking;

[0017] Result output subunit: outputs the judgment decision as the first data analysis result;

[0018] Wherein, a sequential similarity detection algorithm is used in the regional positioning subunit to obtain the best matching point;

[0019] The air pressure judgment standard is: the value when the filling is 1 / 2 completed is used as the first comparison value, and the value when the filling is completed is used as the second comparison value; the first comparison value includes the first air pressure value, the first filling rate and the first air pressure change curve; the second comparison value includes the second air pressure value, the second filling rate and the second air pressure change curve;

[0020] The air pressure data analysis module includes a data analysis unit, which is used to perform data analysis on the processed air pressure detection data, determine whether the packaging bottle is leaking and output a second data analysis result, and specifically includes the following steps:

[0021] Acquire the following data according to the air pressure detection data: a first detection value when the filling is 1 / 2 completed and a second detection value when the filling is completed;

[0022] When the filling is completed 1 / 2, the first detection value is compared with the first comparison value, and if the first detection value is less than the first comparison value, it is determined that the packaging bottle is leaking;

[0023] When the filling is completed, the second detection value is compared with the second comparison value, and if the second detection value is less than the second comparison value, it is determined that the packaging bottle is leaking;

[0024] outputting the judgment result as a second data analysis result;

[0025] Among them, the first detection value includes a first detection air pressure value, a first detection filling rate and a first detection air pressure change curve; the second detection value includes a second detection air pressure value, a second detection filling rate and a second detection air pressure change curve.

[0026] Preferably, the machine vision inspection module includes a vision inspection device configuration unit, a filling inspection unit and an image acquisition unit;

[0027] The visual inspection device configuration unit is used to configure the machine vision inspection device and the inspection environment;

[0028] The filling detection unit is used to detect the filling status of the packaging bottle, and convert the video stream obtained after the detection into a video image and collect it in an image data set;

[0029] The image acquisition unit is used to acquire video images from the image data set and preprocess the video images to generate a preprocessed image set;

[0030] The video image is preprocessed, including the following contents:

[0031] Applying a denoising algorithm to denoise the video image to reduce random noise in the video image;

[0032] Performing image normalization processing on the video image;

[0033] A geometric transformation is performed on the video image.

[0034] Preferably, converting the video stream into a video image includes the following:

[0035] S11, extracting an image of each frame in the video stream;

[0036] S12, processing each frame of the image to obtain a plurality of image frames;

[0037] S13, screening a plurality of image frames using a cyclic continuous structure to obtain the video image;

[0038] The loop continuous structure includes the following contents:

[0039] S131, reading a number of image frames;

[0040] S132, applying an object detection algorithm to analyze image content, classifying image frames with the same features into the same category and forming a plurality of image sets with the same features;

[0041] S133, setting a first similarity threshold and performing similarity judgment on a plurality of image sets with the same characteristics, classifying image sets with the same characteristics that have similar characteristics and whose similarity exceeds the first similarity threshold as similar feature image sets;

[0042] S134, setting a second similarity threshold and selecting all image frames whose image feature similarity exceeds the second similarity threshold from the similar feature image set as video images;

[0043] S135, obtain a new round of image frames and repeat steps S131 to S134.

[0044] Preferably, the image data analysis module further includes an image acquisition unit and an image reprocessing unit;

[0045] The image acquisition unit is used to select a video image from the pre-processed image set as an image to be processed;

[0046] The image reprocessing unit is used to reprocess the image to be processed;

[0047] The image to be processed is reprocessed, including image segmentation and image enhancement:

[0048] The image segmentation includes edge detection and image binarization;

[0049] The image enhancement includes: selecting a mask smoothing algorithm to perform filtering operations, histogram equalization and image contrast linear broadening.

[0050] Preferably, feature analysis and decision making are performed in the feature analysis subunit, specifically including the following steps:

[0051] S51, obtaining an edge contour line obtained after edge detection and performing edge analysis. If the edge contour line is irregular or interrupted, it is determined that the packaging bottle has cracks / damage and the edge analysis result is output;

[0052] S52, acquiring texture features of different regions in the feature region to generate a texture feature vector set and perform texture analysis. If a feature value in the texture feature vector set exceeds a set texture characteristic threshold, it is determined that cracks / damage exist in the packaging bottle and a texture analysis result is output;

[0053] S53, obtaining the number of pixel distributions of the filling liquid feature and the grayscale level of the packaging bottle body and performing pixel distribution comparison. If the number of pixel distributions of the packaging bottle body is greater than or equal to a set liquid grayscale distribution threshold, it is determined that the packaging bottle has liquid leakage and the liquid analysis result is output;

[0054] S54, obtaining a surface image of the packaging bottle and its color distribution characteristics, converting the surface image from the RGB color space to the HSV color space and performing color distribution analysis in the HSV color space. If the color distribution characteristics exceed the set saturation threshold and brightness threshold, it is determined that the packaging bottle has liquid leakage and the color analysis result is output;

[0055] S55, using a support vector machine to make a decision, taking the edge analysis result, texture analysis result, liquid analysis result and color analysis result as inputs of the support vector machine for comprehensive judgment, and outputting a final judgment decision.

[0056] Preferably, texture analysis is performed in step S52, which specifically includes the following steps:

[0057] S521, extracting bottle mouth texture features, bottle body texture features and bottle bottom texture features in the feature area respectively and combining them into a texture feature vector set;

[0058] S522, setting a texture characteristic threshold for determining whether the packaging bottle has cracks / damage;

[0059] S523, selecting a certain eigenvalue from the texture feature vector set and comparing it with the texture characteristic threshold, if the selected eigenvalue exceeds the texture characteristic threshold, executing step S524, otherwise reselecting a eigenvalue for comparison;

[0060] S524, determining whether the packaging bottle has cracks / damage and outputting texture analysis results.

[0061] Preferably, judging whether there is liquid leakage in the packaging bottle in step S53 specifically includes the following steps:

[0062] S531, obtaining the filling liquid characteristics of the filling liquid area and the number of pixels distributed at its grayscale level and setting them as the liquid grayscale distribution threshold;

[0063] S532, obtaining a grayscale histogram of the bottle body before filling and calculating the number of first pixel distributions at the grayscale level;

[0064] S533, obtaining a grayscale histogram of the body of the packaging bottle after filling and calculating the number of second pixel distributions at the grayscale level thereof;

[0065] S534, comparing the first pixel distribution quantity and the second pixel distribution quantity, if the second pixel distribution quantity is greater than or equal to the first pixel distribution quantity, executing step S535; otherwise, determining that there is no liquid leakage in the packaging bottle;

[0066] S535, comparing the second pixel distribution quantity with the liquid grayscale distribution threshold, and if the second pixel distribution quantity is greater than or equal to the liquid grayscale distribution threshold, determining that there is liquid leakage in the packaging bottle;

[0067] In step S54, it is determined whether there is liquid leakage in the packaging bottle, which specifically includes the following steps:

[0068] S541, obtaining a surface image of the filled packaging bottle and a second color distribution feature of the surface image;

[0069] S542, obtaining a first color distribution feature of the surface image of the packaging bottle before filling and using it to set a saturation threshold and a brightness threshold;

[0070] S543, converting the surface image of the filled packaging bottle from the RGB color space to the HSV color space and performing color distribution analysis in the HSV color space; if the saturation of the second color distribution feature exceeds a saturation threshold and the brightness exceeds a brightness threshold, it is determined that liquid leakage exists in the packaging bottle.

[0071] Preferably, the filling air pressure detection module includes a standard construction unit and an air pressure detection unit;

[0072] The standard building unit is used to establish an air pressure judgment standard before performing air pressure detection;

[0073] The air pressure detection unit is used to detect the filling status of the packaging bottle, obtain air pressure detection data and construct an air pressure data set;

[0074] Wherein, establishing the air pressure judgment standard in the standard construction unit specifically includes the following steps:

[0075] S61, obtaining an air pressure reference value and a reference time point when the packaging bottle is empty before air pressure detection;

[0076] S62, obtaining the air pressure value and filling time in the packaging bottle when the filling degree is 1 / 2, and setting them as the first air pressure value and the first time point respectively;

[0077] S63, obtaining the air pressure value and the filling time in the packaging bottle after the filling is completed, and setting them as the second air pressure value and the second time point respectively;

[0078] S64, calculating a first filling rate according to the reference time point and the first time point; calculating a second filling rate according to the first time point and the second time point;

[0079] S65, calculating a first air pressure change curve according to the first air pressure value and the first filling rate; calculating a second air pressure change curve according to the second air pressure value and the second filling rate;

[0080] S66, establish an air pressure judgment standard.

[0081] Preferably, the leakage model building module includes a filling leakage model building unit and a leakage position positioning marking unit;

[0082] The filling leakage model building unit is used to build a filling leakage model according to the first data analysis result and the second data analysis result;

[0083] The leakage position positioning and marking unit is used to determine the leakage position on the packaging bottle according to the filling leakage model and mark it;

[0084] Among them, building a filling leakage model specifically includes the following steps:

[0085] S81, acquiring video image data based on the first data analysis result;

[0086] S82, acquiring air pressure detection data based on the second data analysis result;

[0087] S83, associating the video image data with the air pressure detection data based on the timestamp and forming an associated data set;

[0088] S84: Establish the filling leakage model based on the associated data set.

[0089] In a second aspect, an embodiment of the present application provides a filling status monitoring method, which is applied to a filling status monitoring system as described above, and includes the following steps:

[0090] Use machine vision inspection devices to inspect the filling status of packaging bottles, collect video images and construct preprocessed image sets;

[0091] Acquire the image to be processed for reprocessing and determine whether the packaging bottle is leaking through image analysis, and output a first data analysis result;

[0092] Establish air pressure judgment standards, use air pressure detection devices to detect the filling status of packaging bottles, obtain air pressure detection data and construct an air pressure data set;

[0093] Acquire the data to be processed, perform data processing, and determine whether the packaging bottle is leaking through data analysis, and output a second data analysis result;

[0094] Build a filling leakage model based on the data analysis results and determine the location of the leakage on the packaging bottle;

[0095] Mark the leaking bottles as abnormal products and provide abnormal prompts and abnormal handling;

[0096] Record test results and abnormal conditions, conduct filling evaluation and feedback, and then use this to adjust and control the production process;

[0097] Wherein, the machine vision detection device and the air pressure detection device perform detection and collect data at the same time, thereby generating the same timestamp;

[0098] Perform image analysis on the reprocessed image, including the following contents:

[0099] Obtain the best matching point in the grayscale histogram as the image positioning point and obtain the image offset by calculating the coordinate difference between two consecutive image positioning points, so as to locate according to the image positioning point and move the same distance as the image offset to the feature area;

[0100] Perform feature extraction in the feature area; the feature area includes the bottle mouth area, the bottle body area, the bottle bottom area and the filling liquid area;

[0101] Perform feature analysis on the extracted features and make a decision on whether the packaging bottle is leaking;

[0102] Outputting the judgment decision as a first data analysis result;

[0103] Among them, the sequential similarity detection algorithm is used to obtain the best matching point;

[0104] The air pressure judgment standard is: the value when the filling is 1 / 2 completed is used as the first comparison value, and the value when the filling is completed is used as the second comparison value; the first comparison value includes the first air pressure value, the first filling rate and the first air pressure change curve; the second comparison value includes the second air pressure value, the second filling rate and the second air pressure change curve;

[0105] The processed air pressure detection data is analyzed, specifically including the following steps:

[0106] Acquire the following data according to the air pressure detection data: a first detection value when the filling is 1 / 2 completed and a second detection value when the filling is completed;

[0107] When the filling is completed 1 / 2, the first detection value is compared with the first comparison value, and if the first detection value is less than the first comparison value, it is determined that the packaging bottle is leaking;

[0108] When the filling is completed, the second detection value is compared with the second comparison value, and if the second detection value is less than the second comparison value, it is determined that the packaging bottle is leaking;

[0109] outputting the judgment result as a second data analysis result;

[0110] Among them, the first detection value includes a first detection air pressure value, a first detection filling rate and a first detection air pressure change curve; the second detection value includes a second detection air pressure value, a second detection filling rate and a second detection air pressure change curve.

[0111] The beneficial effects of the present invention are:

[0112] (1) The present invention configures a visual inspection environment and device through a machine vision inspection module, inspects packaging bottles, collects images and performs preprocessing. By converting video streams into video images and adopting a cyclic continuous structure to obtain video images, the video streams captured by the camera can be effectively converted and processed into image data required for machine vision applications, thereby performing various target detection, recognition and analysis tasks, providing data support for subsequent image data analysis; and by classifying the same features and judging the similarity of similar features, the acquired video images are made more complete and rich, which provides data support for subsequent image analysis and processing while also ensuring the accuracy and reliability of image analysis to a certain extent, thereby making the leakage detection of packaging bottles comprehensive and effective.

[0113] (2) The present invention first detects the filling state of the packaging bottle by means of a machine vision detection device, obtains video images and constructs a pre-processed image set, then collects a number of to-be-processed images from the pre-processed image set by means of an image data analysis module for reprocessing, and determines whether the packaging bottle is leaking by means of image analysis and outputs a first data analysis result; further, the filling air pressure detection module uses an air pressure detection device to detect the filling state of the packaging bottle by means of an air pressure detection device, obtains air pressure detection data and constructs an air pressure data set; then, the air pressure data analysis module is used to obtain the to-be-processed data for data processing, and determines whether the packaging bottle is leaking by means of data analysis, and outputs a second data analysis result; then, in the leakage model construction module, a leakage model is constructed based on the first data analysis result and the second data analysis result, and the location of the leakage on the packaging bottle is determined; then, the leakage packaging bottle is marked as an abnormal product by means of an abnormal prompt processing module, and an abnormal prompt and abnormal processing are performed according to the determined leakage location; finally, the above detection results and abnormal conditions are recorded by means of a state recording and feedback module, and the filling conditions are evaluated and fed back, and the production process of the packaging bottle is improved and its production quality is controlled by means of the evaluation and feedback results.

[0114] (3) The present invention uses two different technologies, machine vision technology and air pressure detection technology, to detect the filling status of the packaging bottle respectively, and combines the analysis results of the above two detection technologies in the leakage model construction module to construct a filling leakage model and thereby determine the location where the packaging bottle is leaking, thereby fully combining the advantages of the two detection methods and improving the accuracy and reliability of the detection; then the subsequent abnormal prompt processing module is used to process the leaking packaging bottle and record the detection results for evaluation and feedback, thereby improving the production process and controlling the production quality, comprehensively and efficiently detecting whether the packaging bottle is leaking and promptly processing the leaking packaging bottle, thereby ensuring and improving product quality and production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] For better understanding and implementation, the technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0116] Figure 1 A structural schematic diagram of a filling status monitoring system provided in an embodiment of the present application;

[0117] Figure 2 A schematic diagram of the structure of a machine vision detection module provided in an embodiment of the present application;

[0118] Figure 3 A schematic diagram of the structure of an image data analysis module provided in an embodiment of the present application;

[0119] Figure 4 A schematic diagram of the structure of the air pressure data analysis module provided in an embodiment of the present application;

[0120] Figure 5 A flowchart of the steps of a filling status monitoring method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0121] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of methods and systems consistent with some aspects of the present application as detailed in the attached claims.

[0122] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to any or all possible combinations of one or more associated listed items.

[0123] The specific implementation methods, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0124] Example 1

[0125] See also Figure 1 , the embodiment of the present application provides a filling status monitoring system, including a machine vision detection module, an image data analysis module, a filling air pressure detection module, an air pressure data analysis module, a leakage model building module, an abnormal prompt processing module and a status record feedback module;

[0126] The machine vision detection module uses a machine vision detection device to detect the filling status of the packaging bottle, collect video images and construct a pre-processed image set;

[0127] The image data analysis module is used to obtain the image to be processed, perform reprocessing, and determine whether the packaging bottle is leaking through image analysis, and output a first data analysis result;

[0128] The filling air pressure detection module establishes an air pressure judgment standard, uses an air pressure detection device to detect the filling status of the packaging bottle, obtains air pressure detection data and constructs an air pressure data set;

[0129] The air pressure data analysis module is used to obtain the data to be processed, perform data processing, and determine whether the packaging bottle is leaking through data analysis, and output a second data analysis result;

[0130] The leakage model building module is used to build a filling leakage model based on the data analysis results and determine the location where the leakage occurs on the packaging bottle;

[0131] The abnormality prompt processing module is used to mark the leaking packaging bottle as an abnormal product and perform abnormal prompt and abnormal processing;

[0132] The status recording and feedback module is used to record the test results and abnormal conditions and conduct filling evaluation and feedback, and then adjust and control the production process;

[0133] The machine vision detection device and the air pressure detection device perform detection and collect data at the same time, thereby generating the same timestamp.

[0134] Specifically, in the existing production technology of packaging bottles, there may be problems of leakage of filling materials (mainly liquids) due to various reasons, such as small cracks in the bottle body / bottom of the packaging bottle, insufficient sealing of the bottle mouth, etc. Therefore, in order to ensure the production quality of the packaging bottles, the present application conducts a comprehensive inspection of the filling status of the packaging bottles. The present application first detects the filling state of the packaging bottle by a machine vision detection device, obtains video images and constructs a pre-processed image set, then collects a number of to-be-processed images from the above pre-processed image set by an image data analysis module for reprocessing, and determines whether the packaging bottle is leaking by image analysis and outputs a first data analysis result; the filling state of the packaging bottle is also detected by an air pressure detection device by a filling air pressure detection module, obtains air pressure detection data and constructs an air pressure data set; then the air pressure data analysis module is used to obtain the to-be-processed data for data processing, and determines whether the packaging bottle is leaking by data analysis, and outputs a second data analysis result; then, in a leakage model construction module, a leakage model is constructed based on the above first data analysis result and the second data analysis result, and the location of the leakage on the packaging bottle is determined; then, the leakage packaging bottle is marked as an abnormal product by an abnormal prompt processing module, and an abnormal prompt and abnormal processing are performed according to the determined leakage location; finally, the above detection results and abnormal conditions are recorded by a state recording feedback module, and the filling conditions are evaluated and fed back, and the production process of the packaging bottle is improved and its production quality is controlled by the evaluation and feedback results.

[0135] The present application adopts two different technologies, namely machine vision technology and air pressure detection technology, to detect the filling status of the packaging bottle respectively, and combines the analysis results of the above two detection technologies in the leakage model construction module to construct a filling leakage model and thereby determine the location where the packaging bottle is leaking, which fully combines the advantages of the two detection methods and improves the accuracy and reliability of the detection; and then uses the subsequent abnormal prompt processing module to process the leaking packaging bottle and record the detection results for evaluation and feedback, so as to improve the production process and control the production quality, comprehensively and efficiently detect whether the packaging bottle is leaking and promptly process the leaking packaging bottle, thereby ensuring and improving product quality and production efficiency.

[0136] It should be noted that the above filling state refers to the state of the packaging bottle when it is being filled, and can also be understood as the filling degree, such as: the filling state is the state of the packaging bottle when it is filled to 1 / 2, or the state when the filling is completed. The above abnormal prompt includes a sound warning or an information prompt or an alarm prompt, and the abnormal handling includes destroying the leaking packaging bottle or gluing the gap or treating it as a defective product. This embodiment does not specifically limit the abnormal prompt and abnormal handling.

[0137] See also Figure 2In one embodiment provided in the present application, the machine vision inspection module includes a vision inspection device configuration unit, a filling inspection unit and an image acquisition unit;

[0138] The visual inspection device configuration unit is used to configure the machine vision inspection device and the inspection environment;

[0139] The filling detection unit is used to detect the filling status of the packaging bottle, and convert the video stream obtained after the detection into a video image and collect it in an image data set;

[0140] The image acquisition unit is used to acquire video images from the image data set and preprocess the video images to generate a preprocessed image set;

[0141] The video image is preprocessed, including the following contents:

[0142] Applying a denoising algorithm to denoise the video image to reduce random noise in the video image; the denoising algorithm includes Gaussian blur, median filtering, etc.;

[0143] Performing image normalization processing on the video image, so that it is easier to learn and process image features when the model is used for image processing later, for example, scaling the image pixel values ​​to a fixed range, such as [0, 1] or [-1, 1];

[0144] The video image is geometrically transformed to correct the camera angle, perspective distortion or adjust the image viewing angle, and the geometric transformation includes image rotation, image translation and image scaling.

[0145] Specifically, the function of the machine vision detection module in this embodiment is to configure the environment and device for visual detection, detect the packaging bottle, collect images and perform preprocessing. Through the synergistic effect of the above-mentioned units, the machine vision detection module can provide necessary data support for the subsequent image data analysis module. Among them, the machine vision detection device includes a camera and an imaging device. The camera can be selected with high resolution according to actual needs, and the imaging device is mainly an all-round / comprehensive light source system. The light source system can provide enough light during the visual detection process so that the camera can accurately capture the surface details of the packaging bottle and ensure the imaging quality.

[0146] In an embodiment provided in the present application, converting a video stream into a video image includes the following contents:

[0147] S11, extracting an image of each frame in the video stream; each frame in the video stream represents an instantaneous picture in the video;

[0148] S12, processing each frame of the image to obtain a plurality of image frames; including adjusting the image size, grayscale conversion and color space conversion;

[0149] S13, screening a plurality of image frames using a cyclic continuous structure to obtain the video image;

[0150] The loop continuous structure includes the following contents:

[0151] S131, reading a number of image frames;

[0152] S132, applying an object detection algorithm to analyze image content, classifying image frames with the same features into the same category and forming a plurality of image sets with the same features;

[0153] S133, setting a first similarity threshold and performing similarity judgment on a plurality of image sets with the same characteristics, classifying image sets with the same characteristics that have similar characteristics and whose similarity exceeds the first similarity threshold as similar feature image sets;

[0154] S134, setting a second similarity threshold and selecting all image frames whose image feature similarity exceeds the second similarity threshold from the similar feature image set as video images;

[0155] S135, obtain a new round of image frames and repeat steps S131 to S134.

[0156] Specifically, the present embodiment converts the video stream acquired by the camera into a video image, and its main steps are first to extract the image of each frame in the video stream, then to obtain a number of image frames after adjusting the image size, grayscale conversion and color space conversion, and finally to screen the image frames using a loop continuous structure to obtain a video image; and the loop continuous structure described in the present embodiment refers to the application of an object detection algorithm to analyze the image content after reading the image frame, dividing the image frames with the same features into the same category and forming a plurality of image sets with the same features, and then performing similarity judgment, classifying the image sets with similar features into similar feature image sets, and then selecting all image frames whose image feature similarity exceeds a set second similarity threshold in the similar feature image set as video images, and finally repeating the above steps until a new video image can be obtained after each round of cycles. The present embodiment can effectively convert and process the video stream captured by the camera into image data required for machine vision applications through the above conversion and the use of a loop continuous structure to acquire video images, thereby performing various target detection, recognition and analysis tasks, and providing data support for subsequent image data analysis.

[0157] It should be noted that the same features described in this embodiment refer to that after preliminary feature extraction of the image content, it is found that some features in the image present the same characteristic properties. For example, the features in some image frames all have the characteristics of the packaging bottle mouth, or some image frames all have a certain structure of the packaging bottle body. This embodiment divides the image frames with the same features into the same category and forms multiple sets of images with the same features, which can facilitate subsequent screening, processing and analysis of the images.

[0158] On the other hand, the first similarity threshold set in this embodiment is to group multiple identical feature image sets into an image set with similar features in the subsequent similarity judgment, and the second similarity threshold is set in order to select images from the similar feature image set as video images in the subsequent process; the first similarity threshold can be used to merge identical feature data sets with similar features together, for example: the image features in one identical feature data set are all one side of the packaging bottle, while the image features in another data set are the other side of the packaging bottle, then these two data sets are both one side of the bottle body of the packaging bottle, and therefore have similar feature properties, so the similarity judgment can be made between the two; it should be noted that the specific values ​​of the above two thresholds can be set according to actual conditions or historical data, and this embodiment does not limit their specific values.

[0159] Therefore, this embodiment makes the acquired video images more complete and richer through the classification of the same features and the similarity judgment of similar features, provides data support for subsequent image analysis and processing, and ensures the accuracy and reliability of image analysis to a certain extent.

[0160] See also Figure 3 In one embodiment provided in the present application, the image data analysis module includes an image acquisition unit, an image reprocessing unit and an image analysis unit;

[0161] The image acquisition unit is used to select a video image from the pre-processed image set as an image to be processed;

[0162] The image reprocessing unit is used to reprocess the image to be processed;

[0163] The image analysis unit is used to perform image analysis on the reprocessed image, determine whether the packaging bottle is leaking and output a first data analysis result;

[0164] Wherein, the image obtained after preprocessing by the machine vision detection module is a grayscale histogram;

[0165] The image to be processed is reprocessed, including image segmentation and image enhancement:

[0166] The image segmentation includes edge detection and image binarization, specifically:

[0167] The edge detection comprises the following steps:

[0168] S21, performing smoothing and denoising processing on the image to be processed to reduce noise in the image; Gaussian filtering helps to make the edge detection result more stable and accurate;

[0169] S22, using the Sobel operator to calculate the gradient of the smoothed image to obtain a gradient image; this step is used to detect intensity gradient changes in the image, which usually correspond to the boundaries of the object (packaging bottle); the gradient is also the first-order derivative;

[0170] S23, performing non-maximum suppression on the gradient image; this step can help refine the detected edges, making the edges more detailed and clear;

[0171] S24, applying a dual threshold detection method to detect edges, dividing the gradient values ​​in the gradient image into strong edge pixels, weak edge pixels and non-edge pixels; the dual threshold detection method is: setting two thresholds: a high threshold and a low threshold; the high threshold is used to mark possible strong edge pixels, and the low threshold is used to mark possible weak edge pixels, strong edge pixels are usually directly accepted as final edges, and for weak edge pixels, it is necessary to further determine whether they are connected to strong edge pixels;

[0172] S25, connecting the edges through an edge tracking algorithm, connecting weak edge pixels to strong edge pixels, to form a complete edge line; the edge tracking algorithm of this embodiment can adopt edge tracking based on gradient direction, and this step ensures that the detected edge is a continuous and closed contour.

[0173] S26, performing a thinning process on the detected and connected edges to make them more accurate and smooth, and outputting an edge contour line (or a group of detected edge pixel points) to represent the boundary of the bottle;

[0174] Through the above steps, Canny edge detection can effectively detect the boundaries of the bottle and provide clear object contours, providing the basis and data support for subsequent image segmentation and analysis tasks;

[0175] The image binarization uses an iterative threshold segmentation method to segment the image to be processed, including the following contents:

[0176] S31, select a segmentation threshold T and divide the image into two regions R1 and R2;

[0177] S32, calculating the mean β1 and the mean β2 of the region R1 and the region R2 respectively;

[0178] S33, select a new segmentation threshold T = (β1 + β2) / 2;

[0179] S34, repeat the above steps S31 to S33 until the mean β1 and the mean β2 no longer change;

[0180] The image binarization further uses an adaptive threshold to segment the image to be processed, which is used to automatically determine the threshold according to the grayscale characteristics of the local area of ​​the image, specifically including:

[0181] Determine local areas: divide the image into multiple small local areas;

[0182] Calculate the threshold of each local area: For each local area, calculate a local threshold relative to the area;

[0183] Apply local threshold: Compare the gray value of each pixel with the local threshold of the local area to which it belongs to obtain a local binary image;

[0184] Merge result: Merge the local binary images of all local areas into the final binary image.

[0185] It should be noted that, in practical applications, images may have uneven illumination or large changes in contrast between the target and the background. In this case, the effect of global thresholding may not be ideal. Therefore, this embodiment adopts an adaptive threshold method, which is a method for automatically determining a threshold value based on the grayscale characteristics of a local area of ​​an image, and is suitable for images with uneven grayscale changes or uneven illumination. By using the adaptive threshold method, the threshold is adjusted according to the grayscale level of local pixels, which can solve the problem of discontinuous grayscale changes to a certain extent and adapt to complex backgrounds and non-uniform illumination conditions.

[0186] The image enhancement specifically includes the following contents:

[0187] Select the mask smoothing algorithm for filtering: set several masks with the central pixel (x, y) of the image f(x, y) as the reference point, and calculate the mean of each mask respectively and variance , sort by variance and select the mask with the smallest variance. The grayscale mean of the mask corresponding to the minimum variance is the output value of the central pixel (x, y); then the mean of each mask It is expressed as:

[0188] ;

[0189] Where Q is the number of mask pixels; m and n are the displacements of the pixels in the mask relative to the center pixel (x, y); i is the i-th mask pixel, and i=1,2,...,Q; represents the variance of each mask;

[0190] The image enhancement also includes histogram equalization and linear broadening of image contrast.

[0191] Specifically, edge detection algorithms are used to identify edge information in video images; this helps with subsequent object detection and segmentation; this embodiment combines edge detection and image binarization in image segmentation, specifically including: edge detection can accurately identify the boundaries and contours in the image, and by applying image binarization after edge detection, the pixels at the edge can be segmented into foreground (target) and background, thereby accurately locating the boundaries of the target; secondly, edge detection can suppress noise to a certain extent because it relies on the intensity change between pixels, which helps to reduce the impact of noise during the image binarization process, thereby obtaining a cleaner segmentation result. The method of combining edge detection and image binarization has strong adaptability and is suitable for image segmentation tasks under various lighting conditions and background complexities. In particular, when it is necessary to process images with uneven contrast or complex backgrounds, this method can effectively improve the accuracy and stability of segmentation.

[0192] In image enhancement, a mask smoothing algorithm is used in combination with edge detection and image smoothing processes to improve the results of image segmentation; histogram equalization is used to enhance image contrast by redistributing the intensity values ​​of image pixels; and image contrast linear stretching is used to map the grayscale dynamic range of an image in a certain area to the maximum dynamic range to increase the contrast and brightness of the image.

[0193] Therefore, in this embodiment, the image to be processed is reprocessed in the image data analysis module to ensure the accuracy and reliability of the subsequent analysis image.

[0194] In an embodiment provided in the present application, in the image analysis unit, image analysis is performed on the reprocessed image, specifically including the following contents:

[0195] Region positioning subunit: obtain the best matching point in the grayscale histogram as the image positioning point and obtain the image offset by calculating the coordinate difference between two consecutive image positioning points, so as to locate according to the image positioning point and move the same distance as the image offset to the feature area;

[0196] Feature extraction subunit: performs feature extraction in the feature area; the feature area includes the bottle mouth area, the bottle body area, the bottle bottom area and the filling liquid area;

[0197] Feature analysis subunit: performs feature analysis on the extracted features and makes a decision on whether the packaging bottle is leaking;

[0198] Result output subunit: outputs the judgment decision as the first data analysis result;

[0199] Wherein, in the region positioning subunit, a sequential similarity detection algorithm is used to obtain the best matching point, including the following steps:

[0200] S41, selecting an error criterion as a criterion for terminating the calculation of mismatched points, defining an absolute error value in the error criterion and expressing it as:

[0201] ;

[0202] in, , ; S is the image of the search area, with a size of N×N; T is the reference template, with a size of M×M; is a sub-image;

[0203] S42, set a fixed threshold ;

[0204] S43, in the sub-image Randomly select a point , calculate the absolute error value between it and the corresponding point in the reference template T , the absolute error values ​​corresponding to each random point are accumulated. If the error sum exceeds the set threshold when accumulated to the rth time , then stop accumulating and record the number of accumulations r at this time;

[0205] S44, calculate the error sequence e for the entire grayscale histogram and obtain a surface composed of r values , expressed as:

[0206] ;

[0207] S45, take the surface The position corresponding to the point with the largest median value is taken as the best matching point.

[0208] Specifically, since the packaging bottle may move during the transmission process, the image captured each time is different. Therefore, in order to ensure the accuracy of subsequent feature extraction, the present embodiment sets the above-mentioned area positioning subunit for accurately positioning the packaging bottle, so that the image is within the set feature area each time, providing data support for subsequent feature extraction and analysis; the present embodiment adopts a sequential similarity detection algorithm to obtain the best matching point, and terminates the calculation at the non-matching position by setting a fixed threshold to reduce the amount of calculation, thereby achieving the purpose of improving the operation speed.

[0209] In an embodiment provided in the present application, feature analysis and decision making are performed in the feature analysis subunit, specifically including the following steps:

[0210] S51, obtaining an edge contour line obtained after edge detection and performing edge analysis. If the edge contour line is irregular or interrupted, it is determined that the packaging bottle has cracks / damage and the edge analysis result is output;

[0211] S52, acquiring texture features of different regions in the feature region to generate a texture feature vector set and perform texture analysis. If a feature value in the texture feature vector set exceeds a set texture characteristic threshold, it is determined that cracks / damage exist in the packaging bottle and a texture analysis result is output;

[0212] S53, obtaining the number of pixel distributions of the filling liquid feature and the grayscale level of the packaging bottle body and performing pixel distribution comparison. If the number of pixel distributions of the packaging bottle body is greater than or equal to a set liquid grayscale distribution threshold, it is determined that the packaging bottle has liquid leakage and the liquid analysis result is output;

[0213] S54, obtaining a surface image of the packaging bottle and its color distribution characteristics, converting the surface image from the RGB color space to the HSV color space and performing color distribution analysis in the HSV color space. If the color distribution characteristics exceed the set saturation threshold and brightness threshold, it is determined that the packaging bottle has liquid leakage and the color analysis result is output;

[0214] S55, using a support vector machine to make a decision, taking the edge analysis result, texture analysis result, liquid analysis result and color analysis result as inputs of the support vector machine for comprehensive judgment, and outputting a final judgment decision.

[0215] Specifically, in the feature analysis subunit, the embodiment performs feature analysis from four different angles, namely edge detection, texture analysis, liquid analysis, and bottle surface color analysis, to analyze the leakage or crack / breakage of the packaging bottle from multiple angles, and finally uses a support vector machine to comprehensively analyze the results output by the above four feature analysis means, so as to make a final judgment on whether the packaging bottle has leakage or crack / breakage. The embodiment uses the above multiple feature analysis means for comprehensive analysis and judgment, so that the final judgment decision has higher accuracy, and improves the accuracy and reliability of subsequent data analysis results.

[0216] In an embodiment provided in the present application, texture analysis is performed in step S52, which specifically includes the following steps:

[0217] S521, extracting bottle mouth texture features, bottle body texture features and bottle bottom texture features in the feature area respectively and combining them into a texture feature vector set;

[0218] S522, setting a texture characteristic threshold for determining whether the packaging bottle has cracks / damage; the setting of the texture characteristic threshold can be determined based on actual application requirements and experience, or can be optimized through experiments or training data sets; for example, the threshold can be a numerical value representing a critical point of a certain texture feature;

[0219] S523, selecting a certain eigenvalue from the texture feature vector set and comparing it with the texture characteristic threshold, if the selected eigenvalue exceeds the texture characteristic threshold, executing step S524, otherwise reselecting a eigenvalue for comparison;

[0220] S524, determining whether the packaging bottle has cracks / damage and outputting texture analysis results.

[0221] It should be noted that the above-mentioned texture features can be obtained through methods such as Local Binary Patterns (LBP) and Gray Level Co-occurrence Matrix (GLCM). Therefore, the eigenvalues ​​include contrast and energy extracted from the gray level co-occurrence matrix, as well as the gray level changes of pixels and their neighborhoods described by the local binary patterns.

[0222] In step S53, it is determined whether there is liquid leakage in the packaging bottle, which specifically includes the following steps:

[0223] S531, obtaining the filling liquid characteristics of the filling liquid area and the number of pixels distributed at its grayscale level and setting them as the liquid grayscale distribution threshold;

[0224] S532, obtaining a grayscale histogram of the bottle body before filling and calculating the number of first pixel distributions at the grayscale level;

[0225] S533, obtaining a grayscale histogram of the body of the packaging bottle after filling and calculating the number of second pixel distributions at the grayscale level thereof;

[0226] S534, comparing the first pixel distribution quantity and the second pixel distribution quantity, if the second pixel distribution quantity is greater than or equal to the first pixel distribution quantity, executing step S535; otherwise, determining that there is no liquid leakage in the packaging bottle;

[0227] S535, comparing the second pixel distribution quantity with the liquid grayscale distribution threshold, if it is greater than or equal to the liquid grayscale distribution threshold, it is determined that there is liquid leakage in the packaging bottle.

[0228] It should be noted that the packaging bottle is empty before filling, and the grayscale histogram of the bottle body can clearly show the empty bottle state at this time; after filling, because of the addition of filling liquid, the grayscale histogram of the packaging bottle body will be obviously different from that before filling; and the pixel distribution numbers of the grayscale levels of the bottle body after filling and before filling are compared. Since the addition of liquid after filling will cause the grayscale level of the bottle body to increase, that is, the number of pixel distributions increases, it can be clearly judged that there may be leaking liquid on the bottle body after filling. However, in order to ensure the accuracy of the judgment, this embodiment further judges by judging the pixel distribution number after filling and the liquid grayscale distribution threshold of the filling liquid area. When the threshold is exceeded, it means that the pixel distribution number on the bottle body after filling is greater than the pixel distribution number of the filling liquid. Therefore, it can be proved that there is filling liquid on the bottle body, thereby judging that liquid leakage has occurred in the packaging bottle.

[0229] In step S54, it is determined whether there is liquid leakage in the packaging bottle, which specifically includes the following steps:

[0230] S541, obtaining a surface image of the filled packaging bottle and a second color distribution feature of the surface image;

[0231] S542, obtaining a first color distribution feature of the surface image of the packaging bottle before filling and using it to set a saturation threshold and a brightness threshold;

[0232] S543, converting the surface image of the filled packaging bottle from the RGB color space to the HSV color space and performing color distribution analysis in the HSV color space; if the saturation of the second color distribution feature exceeds a saturation threshold and the brightness exceeds a brightness threshold, it is determined that liquid leakage exists in the packaging bottle.

[0233] It should be noted that, since the method for judging whether there is liquid leakage in the packaging bottle in the above step S53 adopts the grayscale histogram of the packaging bottle and its grayscale level and pixel distribution number, but since the grayscale histogram may lead to misjudgment, in order to ensure the accuracy of the judgment, this embodiment adopts the color distribution of the surface image of the packaging bottle before and after filling to judge whether there is leakage.

[0234] To sum up, in the above-mentioned feature analysis subunit, this embodiment adopts four different analysis methods to determine whether the packaging bottle is leaking, and jointly generates the final decision based on the four judgment results, thereby avoiding wrong judgment or misjudgment caused by a single analysis method, and greatly improving the accuracy and reliability of the final decision to a certain extent.

[0235] In one embodiment provided in the present application, the filling air pressure detection module includes a standard construction unit and an air pressure detection unit;

[0236] The standard building unit is used to establish an air pressure judgment standard before performing air pressure detection;

[0237] The air pressure detection unit is used to detect the filling status of the packaging bottle, obtain air pressure detection data and construct an air pressure data set;

[0238] Wherein, establishing the air pressure judgment standard in the standard construction unit specifically includes the following steps:

[0239] S61, obtaining an air pressure reference value and a reference time point when the packaging bottle is empty before air pressure detection;

[0240] S62, obtaining the air pressure value and filling time in the packaging bottle when the filling degree is 1 / 2, and setting them as the first air pressure value and the first time point respectively;

[0241] S63, obtaining the air pressure value and the filling time in the packaging bottle after the filling is completed, and setting them as the second air pressure value and the second time point respectively;

[0242] S64, calculating a first filling rate according to the reference time point and the first time point; calculating a second filling rate according to the first time point and the second time point;

[0243] S65, calculating a first air pressure change curve according to the first air pressure value and the first filling rate; calculating a second air pressure change curve according to the second air pressure value and the second filling rate;

[0244] S66, establishing an air pressure judgment standard: taking the value when the filling is 1 / 2 completed as the first comparison value, and taking the value when the filling is completed as the second comparison value;

[0245] The first comparison value includes a first air pressure value, a first filling rate and a first air pressure change curve; the second comparison value includes a second air pressure value, a second filling rate and a second air pressure change curve.

[0246] Specifically, before the air pressure detection is performed in this embodiment, the air pressure judgment standard is first established based on the intact empty bottle and the air pressure data change obtained by filling the empty bottle. The standard describes the standard value and change range of the air pressure of this type of packaging bottle when it is intact, and can be used as a judgment standard for subsequent detection of whether the packaging bottle has cracks / breakages; and the first air pressure change curve and the second air pressure change curve respectively represent the changes in air pressure value and filling rate under different filling degrees, and this embodiment does not limit the specific expression method; this embodiment uses the change in air pressure value in the packaging bottle when the filling degree is 1 / 2 as the judgment condition and standard to better feedback the air pressure change, and can more quickly determine the location of the leakage. In addition, the above-mentioned filling rate refers to the change in pressure per unit time, so in this embodiment, it can be calculated by time point and air pressure value.

[0247] It should be noted that, in the filling air pressure detection module of the present application, an air pressure detection device is used to detect the filling status of the packaging bottle. The above-mentioned air pressure detection device includes a sensor, a pressure gauge, a data acquisition system, etc. This embodiment does not limit the air pressure detection device.

[0248] See also Figure 4 In one embodiment provided in the present application, the air pressure data analysis module includes a data acquisition unit, a data processing unit and a data analysis unit;

[0249] The data acquisition unit selects and acquires air pressure detection data from the air pressure data set as data to be processed;

[0250] The data processing unit is used to process the data to be processed;

[0251] The data analysis unit is used to perform data analysis on the processed air pressure detection data, determine whether the packaging bottle is leaking, and output a second data analysis result;

[0252] The data analysis of the air pressure detection data specifically includes the following steps:

[0253] S71, acquiring the following data according to the air pressure detection data: a first detection value when the filling is 1 / 2 completed and a second detection value when the filling is completed;

[0254] S72, when the filling is completed 1 / 2, comparing the first detection value with the first comparison value, if the first detection value is less than the first comparison value, it is determined that the packaging bottle is leaking;

[0255] S73, when the filling is completed, comparing the second detection value with the second comparison value, if the second detection value is less than the second comparison value, it is determined that the packaging bottle is leaking;

[0256] S74, outputting the determination result as a second data analysis result;

[0257] Among them, the first detection value includes a first detection air pressure value, a first detection filling rate and a first detection air pressure change curve; the second detection value includes a second detection air pressure value, a second detection filling rate and a second detection air pressure change curve.

[0258] Specifically, since image processing and image analysis may have certain errors and misjudgments, in order to further ensure the accuracy of the judgment results, this embodiment, on the basis of the above-mentioned machine vision detection and analysis, also uses air pressure detection technology to detect the filling air pressure, and compares the detected air pressure value, filling rate and air pressure change curve with the comparison value in the air pressure judgment standard established before filling in the data analysis unit. When the detection value is less than the comparison value, it means that the air pressure in the bottle is less than the standard air pressure in the bottle under normal and non-destructive conditions, that is, the detected packaging bottle has cracks or damage. The two filling levels set in this embodiment, when the filling is completed 1 / 2 and when the filling is completed, can judge the change of air pressure in the bottle in stages, which is conducive to the subsequent determination of the leakage position. For example, if the first detection value is less than the first comparison value when the filling is completed 1 / 2, the leakage position of the packaging bottle is at 1 / 2 of the packaging bottle.

[0259] In one embodiment provided in the present application, the leakage model building module includes a filling leakage model building unit and a leakage position positioning marking unit;

[0260] The filling leakage model building unit is used to build a filling leakage model according to the first data analysis result and the second data analysis result;

[0261] The leakage position positioning and marking unit is used to determine the leakage position on the packaging bottle according to the filling leakage model and mark it;

[0262] Among them, building a filling leakage model specifically includes the following steps:

[0263] S81, acquiring video image data based on the first data analysis result;

[0264] S82, acquiring air pressure detection data based on the second data analysis result;

[0265] S83, associating the video image data with the air pressure detection data based on the timestamp and forming an associated data set;

[0266] S84: Establish the filling leakage model based on the associated data set.

[0267] Specifically, in the leakage model building module, the present embodiment obtains the video image data and the air pressure detection data respectively according to the first data analysis result output by the image data analysis module and the second data analysis result output by the air pressure data analysis module, and associates the two data through the timestamp, and then combines the judgment results of the above two data analysis results (such as whether the packaging bottle is leaking and the specific location of the leakage, etc.) as key data to build a filling leakage model, and uses the model to infer the filling abnormality and pressure change, so as to determine the location and cause of the leakage of the packaging bottle. It should be noted that the present application uses the machine vision detection device and the air pressure detection device to detect and collect data at the same time, so the two can generate the same timestamp, and the timestamp can establish an association relationship for the above video image data and air pressure detection data, so that the two have correlation, rather than two independent types of data. The present embodiment effectively associates the data obtained from the machine vision and air pressure detection through the above modules, thereby providing accurate and reliable data support for the filling state and filling quality of the packaging bottle during the production process.

[0268] Example 2

[0269] See also Figure 5 The present application embodiment provides a filling status monitoring method, comprising the following steps:

[0270] Use machine vision inspection devices to inspect the filling status of packaging bottles, collect video images and construct preprocessed image sets;

[0271] Acquire the image to be processed for reprocessing and determine whether the packaging bottle is leaking through image analysis, and output a first data analysis result;

[0272] Establish air pressure judgment standards, use air pressure detection devices to detect the filling status of packaging bottles, obtain air pressure detection data and construct an air pressure data set;

[0273] Acquire the data to be processed, perform data processing, and determine whether the packaging bottle is leaking through data analysis, and output a second data analysis result;

[0274] Build a filling leakage model based on the data analysis results and determine the location of the leakage on the packaging bottle;

[0275] Mark the leaking bottles as abnormal products and provide abnormal prompts and abnormal handling;

[0276] Record test results and abnormal conditions, conduct filling evaluation and feedback, and then use this to adjust and control the production process;

[0277] Wherein, the machine vision detection device and the air pressure detection device perform detection and collect data at the same time, thereby generating the same timestamp;

[0278] Perform image analysis on the reprocessed image, including the following contents:

[0279] Obtain the best matching point in the grayscale histogram as the image positioning point and obtain the image offset by calculating the coordinate difference between two consecutive image positioning points, so as to locate according to the image positioning point and move the same distance as the image offset to the feature area;

[0280] Perform feature extraction in the feature area; the feature area includes the bottle mouth area, the bottle body area, the bottle bottom area and the filling liquid area;

[0281] Perform feature analysis on the extracted features and make a decision on whether the packaging bottle is leaking;

[0282] Outputting the judgment decision as a first data analysis result;

[0283] Among them, the sequential similarity detection algorithm is used to obtain the best matching point;

[0284] The air pressure judgment standard is: the value when the filling is 1 / 2 completed is used as the first comparison value, and the value when the filling is completed is used as the second comparison value; the first comparison value includes the first air pressure value, the first filling rate and the first air pressure change curve; the second comparison value includes the second air pressure value, the second filling rate and the second air pressure change curve;

[0285] The processed air pressure detection data is analyzed, specifically including the following steps:

[0286] Acquire the following data according to the air pressure detection data: a first detection value when the filling is 1 / 2 completed and a second detection value when the filling is completed;

[0287] When the filling is completed 1 / 2, the first detection value is compared with the first comparison value, and if the first detection value is less than the first comparison value, it is determined that the packaging bottle is leaking;

[0288] When the filling is completed, the second detection value is compared with the second comparison value, and if the second detection value is less than the second comparison value, it is determined that the packaging bottle is leaking;

[0289] outputting the judgment result as a second data analysis result;

[0290] Among them, the first detection value includes a first detection air pressure value, a first detection filling rate and a first detection air pressure change curve; the second detection value includes a second detection air pressure value, a second detection filling rate and a second detection air pressure change curve.

[0291] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0292] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0293] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0294] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A filling status monitoring system, characterized in that: Including machine vision detection module and image data analysis module; The machine vision detection module uses a machine vision detection device to detect the filling status of the packaging bottle, collect video images and construct a pre-processed image set; The image data analysis module is used to obtain the image to be processed, perform reprocessing, and determine whether the packaging bottle is leaking through image analysis, and output a first data analysis result; The machine vision inspection module includes a vision inspection device configuration unit, a filling inspection unit and an image acquisition unit; The visual inspection device configuration unit is used to configure the machine vision inspection device and the inspection environment; The filling detection unit is used to detect the filling status of the packaging bottle, and convert the video stream obtained after the detection into a video image and collect it in an image data set; The image acquisition unit is used to acquire video images from the image data set and preprocess the video images to generate a preprocessed image set; The video image is preprocessed, including the following contents: Applying a denoising algorithm to denoise the video image to reduce random noise in the video image; Performing image normalization processing on the video image; Performing geometric transformation on the video image; The video stream is converted into a video image, including the following contents: S11, extracting an image of each frame in the video stream; S12, processing each frame of the image to obtain a plurality of image frames; S13, screening a plurality of image frames using a cyclic continuous structure to obtain the video image; The loop continuous structure includes the following contents: S131, reading a number of image frames; S132, applying an object detection algorithm to analyze image content, classifying image frames with the same features into the same category and forming a plurality of image sets with the same features; S133, setting a first similarity threshold and performing similarity judgment on a plurality of image sets with the same characteristics, classifying image sets with the same characteristics that have similar characteristics and whose similarity exceeds the first similarity threshold as similar feature image sets; S134, setting a second similarity threshold and selecting all image frames whose image feature similarity exceeds the second similarity threshold from the similar feature image set as video images; S135, acquiring a new round of image frames and repeating steps S131 to S134; The filling status monitoring system also includes a filling air pressure detection module, an air pressure data analysis module, a leakage model building module, an abnormal prompt processing module and a status record feedback module. The filling air pressure detection module establishes an air pressure judgment standard, uses an air pressure detection device to detect the filling status of the packaging bottle, obtains air pressure detection data and constructs an air pressure data set; The air pressure data analysis module is used to obtain the data to be processed, perform data processing, and determine whether the packaging bottle is leaking through data analysis, and output a second data analysis result; The leakage model building module builds a filling leakage model based on the data analysis results and determines the location where the leakage occurs on the packaging bottle; The abnormality prompt processing module is used to mark the leaking packaging bottle as an abnormal product and perform abnormal prompt and abnormal processing; The status recording and feedback module is used to record the test results and abnormal conditions and conduct filling evaluation and feedback, and then adjust and control the production process; Wherein, the machine vision detection device and the air pressure detection device perform detection and collect data at the same time, thereby generating the same timestamp; The air pressure judgment standard is: the value when the filling is 1 / 2 completed is used as the first comparison value, and the value when the filling is completed is used as the second comparison value; the first comparison value includes the first air pressure value, the first filling rate and the first air pressure change curve; the second comparison value includes the second air pressure value, the second filling rate and the second air pressure change curve; The air pressure data analysis module includes a data analysis unit, which is used to perform data analysis on the processed air pressure detection data, determine whether the packaging bottle is leaking and output a second data analysis result, and specifically includes the following steps: Acquire the following data according to the air pressure detection data: a first detection value when the filling is 1 / 2 completed and a second detection value when the filling is completed; When the filling is completed 1 / 2, the first detection value is compared with the first comparison value, and if the first detection value is less than the first comparison value, it is determined that the packaging bottle is leaking; When the filling is completed, the second detection value is compared with the second comparison value, and if the second detection value is less than the second comparison value, it is determined that the packaging bottle is leaking; outputting the judgment result as a second data analysis result; Among them, the first detection value includes a first detection air pressure value, a first detection filling rate and a first detection air pressure change curve; the second detection value includes a second detection air pressure value, a second detection filling rate and a second detection air pressure change curve.

2. A filling status monitoring system according to claim 1, characterized in that: The image data analysis module includes an image analysis unit, which is used to perform image analysis on the reprocessed image, determine whether the packaging bottle leaks and output a first data analysis result, which specifically includes the following contents: Region positioning subunit: obtain the best matching point in the grayscale histogram as the image positioning point and obtain the image offset by calculating the coordinate difference between two consecutive image positioning points, so as to locate according to the image positioning point and move the same distance as the image offset to the feature area; Feature extraction subunit: performs feature extraction in the feature area; the feature area includes the bottle mouth area, the bottle body area, the bottle bottom area and the filling liquid area; Feature analysis subunit: performs feature analysis on the extracted features and makes a decision on whether the packaging bottle is leaking; Result output subunit: outputs the judgment decision as the first data analysis result; Wherein, a sequential similarity detection algorithm is used in the regional positioning subunit to obtain the best matching point; The image data analysis module also includes an image acquisition unit and an image reprocessing unit; The image acquisition unit is used to select a video image from the pre-processed image set as an image to be processed; The image reprocessing unit is used to reprocess the image to be processed; The image to be processed is reprocessed, including image segmentation and image enhancement: The image segmentation includes edge detection and image binarization; The image enhancement includes: selecting a mask smoothing algorithm to perform filtering operations, histogram equalization and image contrast linear broadening.

3. A filling status monitoring system according to claim 2, characterized in that: The feature analysis and decision making are performed in the feature analysis subunit, specifically including the following steps: S51, obtaining an edge contour line obtained after edge detection and performing edge analysis. If the edge contour line is irregular or interrupted, it is determined that the packaging bottle has cracks / damage and the edge analysis result is output; S52, acquiring texture features of different regions in the feature region to generate a texture feature vector set and perform texture analysis. If a feature value in the texture feature vector set exceeds a set texture characteristic threshold, it is determined that cracks / damage exist in the packaging bottle and a texture analysis result is output; S53, obtaining the number of pixel distributions of the filling liquid feature and the grayscale level of the packaging bottle body and performing pixel distribution comparison. If the number of pixel distributions of the packaging bottle body is greater than or equal to a set liquid grayscale distribution threshold, it is determined that the packaging bottle has liquid leakage and the liquid analysis result is output; S54, obtaining a surface image of the packaging bottle and its color distribution characteristics, converting the surface image from the RGB color space to the HSV color space and performing color distribution analysis in the HSV color space. If the color distribution characteristics exceed the set saturation threshold and brightness threshold, it is determined that the packaging bottle has liquid leakage and the color analysis result is output; S55, using a support vector machine to make a decision, taking the edge analysis result, texture analysis result, liquid analysis result and color analysis result as inputs of the support vector machine for comprehensive judgment, and outputting a final judgment decision.

4. A filling status monitoring system according to claim 3, characterized in that: In step S52, texture analysis is performed, which specifically includes the following steps: S521, extracting bottle mouth texture features, bottle body texture features and bottle bottom texture features in the feature area respectively and combining them into a texture feature vector set; S522, setting a texture characteristic threshold for determining whether the packaging bottle has cracks / damage; S523, selecting a certain eigenvalue from the texture feature vector set and comparing it with the texture characteristic threshold, if the selected eigenvalue exceeds the texture characteristic threshold, executing step S524, otherwise reselecting a eigenvalue for comparison; S524, determining whether the packaging bottle has cracks / damage and outputting texture analysis results.

5. A filling status monitoring system according to claim 3, characterized in that: In step S53, it is determined whether there is liquid leakage in the packaging bottle, which specifically includes the following steps: S531, obtaining the filling liquid characteristics of the filling liquid area and the number of pixels distributed at its grayscale level and setting them as the liquid grayscale distribution threshold; S532, obtaining a grayscale histogram of the bottle body before filling and calculating the number of first pixel distributions at the grayscale level; S533, obtaining a grayscale histogram of the body of the packaging bottle after filling and calculating the number of second pixel distributions at the grayscale level thereof; S534, comparing the first pixel distribution quantity and the second pixel distribution quantity, if the second pixel distribution quantity is greater than or equal to the first pixel distribution quantity, executing step S535; otherwise, determining that there is no liquid leakage in the packaging bottle; S535, comparing the second pixel distribution quantity with the liquid grayscale distribution threshold, and if the second pixel distribution quantity is greater than or equal to the liquid grayscale distribution threshold, determining that there is liquid leakage in the packaging bottle; In step S54, it is determined whether there is liquid leakage in the packaging bottle, which specifically includes the following steps: S541, obtaining a surface image of the filled packaging bottle and a second color distribution feature of the surface image; S542, obtaining a first color distribution feature of the surface image of the packaging bottle before filling and using it to set a saturation threshold and a brightness threshold; S543, converting the surface image of the filled packaging bottle from the RGB color space to the HSV color space and performing color distribution analysis in the HSV color space; if the saturation of the second color distribution feature exceeds a saturation threshold and the brightness exceeds a brightness threshold, it is determined that liquid leakage exists in the packaging bottle.

6. A filling status monitoring system according to claim 1, characterized in that: The filling air pressure detection module includes a standard construction unit and an air pressure detection unit; The standard building unit is used to establish an air pressure judgment standard before performing air pressure detection; The air pressure detection unit is used to detect the filling status of the packaging bottle, obtain air pressure detection data and construct an air pressure data set; Wherein, establishing the air pressure judgment standard in the standard construction unit specifically includes the following steps: S61, obtaining an air pressure reference value and a reference time point when the packaging bottle is empty before air pressure detection; S62, obtaining the air pressure value and filling time in the packaging bottle when the filling degree is 1 / 2, and setting them as the first air pressure value and the first time point respectively; S63, obtaining the air pressure value and the filling time in the packaging bottle after the filling is completed, and setting them as the second air pressure value and the second time point respectively; S64, calculating a first filling rate according to the reference time point and the first time point; calculating a second filling rate according to the first time point and the second time point; S65, calculating a first air pressure change curve according to the first air pressure value and the first filling rate; calculating a second air pressure change curve according to the second air pressure value and the second filling rate; S66, establish an air pressure judgment standard.

7. A filling status monitoring system according to claim 1, characterized in that: The leakage model building module includes a filling leakage model building unit and a leakage position positioning marking unit; The filling leakage model building unit is used to build a filling leakage model according to the first data analysis result and the second data analysis result; The leakage position positioning and marking unit is used to determine the leakage position on the packaging bottle according to the filling leakage model and mark it; Among them, building a filling leakage model specifically includes the following steps: S81, acquiring video image data based on the first data analysis result; S82, acquiring air pressure detection data based on the second data analysis result; S83, associating the video image data with the air pressure detection data based on the timestamp and forming an associated data set; S84, establishing the filling leakage model based on the associated data set.

8. A filling status monitoring method, applied to a filling status monitoring system as claimed in any one of claims 1 to 7, characterized in that: The steps include: The filling status of the packaging bottle is detected by a machine vision inspection device, video images are collected and a pre-processed image set is constructed, specifically: Configure machine vision inspection equipment and inspection environment; Detect the filling status of the packaging bottle, convert the video stream obtained after the detection into video images and collect them in the image dataset; Acquire video images from the image data set and preprocess the video images to generate a preprocessed image set; The video image is preprocessed, including the following contents: Applying a denoising algorithm to denoise the video image to reduce random noise in the video image; Performing image normalization processing on the video image; Performing geometric transformation on the video image; The video stream is converted into a video image, including the following contents: S11, extracting an image of each frame in the video stream; S12, processing each frame of the image to obtain a plurality of image frames; S13, screening a plurality of image frames using a cyclic continuous structure to obtain the video image; The loop continuous structure includes the following contents: S131, reading a number of image frames; S132, applying an object detection algorithm to analyze image content, classifying image frames with the same features into the same category and forming a plurality of image sets with the same features; S133, setting a first similarity threshold and performing similarity judgment on a plurality of image sets with the same characteristics, classifying image sets with the same characteristics that have similar characteristics and whose similarity exceeds the first similarity threshold as similar feature image sets; S134, setting a second similarity threshold and selecting all image frames whose image feature similarity exceeds the second similarity threshold from the similar feature image set as video images; S135, acquiring a new round of image frames and repeating steps S131 to S134; The image to be processed is obtained for reprocessing and whether leakage occurs in the packaging bottle is determined through image analysis, and a first data analysis result is output.

Images