A method for recognizing a bag opening during filling of a beverage bag

Abstract

The application provides a bag mouth recognition method during beverage bag filling, comprising: obtaining a bag body image of a beverage bag, and recognizing a bag body region and a suction nozzle candidate region on the bag body; obtaining three-dimensional structure information of the suction nozzle candidate region, and recognizing a real position of the suction nozzle to obtain initial positioning coordinates of the suction nozzle; extracting shoulder feature points of the bag body region, constructing a dynamic constraint model based on a spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the suction nozzle, and verifying and correcting the initial positioning coordinates of the suction nozzle by using the dynamic constraint model to obtain corrected positioning coordinates of the suction nozzle; collecting corrected positioning coordinates of the same suction nozzle in a continuous predetermined number of frames of images to form a motion trajectory of the suction nozzle, and performing position prediction on the motion trajectory of the suction nozzle based on a time sequence filtering algorithm to output final positioning coordinates of the suction nozzle. The recognition method provided by the application realizes stable and accurate positioning of a flexible bag suction nozzle by fusing image analysis, three-dimensional structure recognition and shoulder topological constraints.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method for identifying the bag opening during beverage bag filling. Background Technology

[0002] In beverage production, the filling process is a crucial step determining product quality and production efficiency. Filling equipment needs to accurately identify the container opening position to precisely insert the filling head for filling and subsequent sealing. Traditional filling positioning often uses mechanical methods, limiting the container's position with physical blocks or guides. However, this method has low positioning accuracy and is unsuitable for high-speed filling production. With the development of machine vision technology, more and more filling equipment is adopting vision positioning systems. By acquiring container images and performing image processing and analysis, it achieves non-contact, precise identification of the container opening position.

[0003] Existing visual positioning methods typically extract and identify features directly from images of the container opening to obtain its spatial coordinates. However, these methods have significant limitations when applied to flexible beverage bags. Beverage bags are made of flexible material, and their posture changes randomly during transport. Only the nozzle position remains relatively fixed, while there is no stable spatial correspondence between other parts of the bag and the nozzle. Existing visual positioning methods are susceptible to changes in ambient lighting, reflections from the bag surface, and water mist interference, leading to decreased nozzle recognition accuracy and consequently affecting the reliability of filling and sealing operations. Furthermore, because the visual features of the nozzle in an image are relatively simple, when there are similarly shaped interfering objects or local features are obscured, relying solely on the image information of the nozzle itself is insufficient to guarantee stable positioning. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, a method for identifying the bag opening during beverage bag filling is provided, comprising: Acquire an image of the beverage bag on the conveying device, and analyze the image to identify the bag area and the candidate area for the spout on the bag. The three-dimensional structural information of the candidate region of the suction nozzle is obtained, and the actual position of the suction nozzle is identified based on the three-dimensional structural information to obtain the preliminary positioning coordinates of the suction nozzle; Extract shoulder feature points from the bag area, construct a dynamic constraint model based on the spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the nozzle, and use the dynamic constraint model to verify and correct the initial positioning coordinates of the nozzle to obtain the corrected positioning coordinates of the nozzle. The corrected positioning coordinates of the same nozzle are acquired in a predetermined number of consecutive frames of images to form the nozzle motion trajectory. The position of the nozzle motion trajectory is predicted based on a time series filtering algorithm, and the final positioning coordinates of the nozzle are output.

[0005] According to the technical solution provided by the present invention, the step of analyzing the bag image to identify the bag area and the candidate area for the suction nozzle on the bag includes: The bag image is input into a pre-trained deep learning instance segmentation model to obtain a bag mask and a nozzle candidate region mask. The bag outline is extracted based on the bag mask, and shoulder feature points are extracted in the upper half of the bag outline, including left shoulder feature points and right shoulder feature points; The position of the nozzle candidate region is determined based on the nozzle candidate region mask.

[0006] According to the technical solution provided by the present invention, the step of obtaining the three-dimensional structural information of the nozzle candidate region and identifying the true position of the nozzle based on the three-dimensional structural information includes: The structured light projection unit is controlled to project coded structured light onto the candidate area of ​​the suction nozzle, and images with structured light stripes are acquired simultaneously. The image with structured light stripes is encoded and phase calculated to reconstruct the three-dimensional point cloud data of the nozzle candidate region; The three-dimensional point cloud data is filtered, denoised, and clustered to obtain multiple point cloud clusters; Geometric feature analysis is performed on each point cloud cluster. The geometric features include size, shape, surface normal vector variation, and the degree of concavity at the connection between the bottom and the surrounding area. Point cloud clusters that meet the rigid geometric features of the nozzle and whose concavity exceeds a preset concavity threshold are identified as real nozzles. The geometric center of the point cloud cluster is used as the initial positioning coordinates of the nozzle.

[0007] According to the technical solution provided by the present invention, the step of constructing a dynamic constraint model based on the spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the suction nozzle, and using the dynamic constraint model to verify and correct the initial positioning coordinates of the suction nozzle, includes: The two-dimensional image coordinates of the left and right shoulder feature points are converted into estimated three-dimensional spatial coordinates. Calculate the first spatial distance between the initial positioning coordinates of the suction nozzle and the feature point of the left shoulder, the second spatial distance between the nozzle and the feature point of the right shoulder, and the spatial angle between the first vector pointing from the suction nozzle to the feature point of the left shoulder and the second vector pointing from the suction nozzle to the feature point of the right shoulder; The first spatial distance, the second spatial distance, and the spatial angle are used as the current measured values ​​and compared with the expected values ​​maintained by the dynamic constraint model to calculate the degree of deviation between the current measured values ​​and the expected values. The correction positioning coordinates of the nozzle are determined by verifying the deviation degree against the preset deviation threshold. The expected value and variance of the dynamic constraint model are updated online using the validated current measurements.

[0008] According to the technical solution provided by the present invention, the step of verifying and determining the corrected positioning coordinates of the nozzle based on the comparison result of the deviation degree and the preset deviation threshold includes: If the deviation is less than the first preset deviation threshold, the initial positioning coordinates of the nozzle are directly used as the corrected positioning coordinates of the nozzle. If the deviation is greater than or equal to the first preset deviation threshold and less than the second preset deviation threshold, then the initial positioning coordinates of the nozzle and the predicted position predicted by the dynamic constraint model are weighted and fused to obtain the corrected positioning coordinates of the nozzle. If the deviation is greater than or equal to the second preset deviation threshold, the initial positioning coordinates of the nozzle are discarded, and the predicted position predicted by the dynamic constraint model is used as the corrected positioning coordinates of the nozzle.

[0009] According to the technical solution provided by the present invention, the corrected positioning coordinates of the same suction nozzle are acquired in a predetermined number of consecutive frames of images to form a suction nozzle motion trajectory, and the position of the suction nozzle motion trajectory is predicted based on a time series filtering algorithm, including: Construct a state vector and a state transition model. The state vector includes the position component and velocity component of the nozzle. The state transition model is used to describe the change law of the state vector. Based on the posterior state estimate of the previous frame and the state transition model, the prior state estimate of the current frame is predicted. The nozzle correction positioning coordinates of the current frame are fused with the prior state estimate as the observation value to obtain the posterior state estimate of the current frame, and the position component is extracted from the posterior state estimate as the final positioning coordinates of the nozzle. The position of the nozzle in the next frame is predicted based on the posterior state estimation, and the prediction result is sent to the actuator for look-ahead control.

[0010] According to the technical solution provided by the present invention, the process of analyzing the bag image further includes: If the candidate area for the suction nozzle cannot be identified, the bag image is re-acquired and re-identified. If the retry fails, the nozzle position of the next frame predicted by the time series filtering algorithm will be used as the final positioning coordinates of the current frame. If the candidate region for the sip nozzle cannot be identified for three consecutive frames, the detection threshold of the deep learning instance segmentation model is lowered or the backup detection algorithm is switched. If the detection still fails, an alarm signal is issued and the filling operation of the current beverage bag is paused.

[0011] According to the technical solution provided by the present invention, the process of obtaining the three-dimensional structural information of the nozzle candidate region and identifying the true position of the nozzle based on the three-dimensional structural information further includes: If the actual nozzle cannot be identified from the 3D point cloud data, adjust the structured light projection intensity, re-acquire images with structured light stripes, and reconstruct the 3D point cloud data. If the retry fails, the three-dimensional position of the nozzle is estimated based on the two-dimensional image coordinates of the nozzle candidate area and the historical average depth data of the bag area, and the estimated position coordinates are used for subsequent verification and correction.

[0012] According to the technical solution provided by the present invention, in the process of constructing a dynamic constraint model based on the spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the suction nozzle, and using the dynamic constraint model to verify and correct the initial positioning coordinates of the suction nozzle, the method further includes: If an abnormal situation is detected in which three consecutive frames show a deviation greater than or equal to the second preset deviation threshold, the expected value and variance of the dynamic constraint model are reinitialized, and the operator is prompted to confirm whether the bag type or the status of the conveying device has changed.

[0013] According to the technical solution provided by the present invention, it also includes: Calculate the difference between the current frame observation and the prior state estimate; If the difference exceeds a preset threshold for three consecutive frames, the process noise covariance of the state transition model is increased, and the filling speed is reduced or an alarm signal is issued.

[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: First, the method identifies candidate areas for the suction nozzle through bag image analysis, providing an initial search range for subsequent precise positioning and avoiding blind searching across the entire image. Based on this, the true position of the suction nozzle is identified by acquiring the three-dimensional structural information of the candidate areas, effectively overcoming the influence of ambient light variations, bag surface reflections, and water mist interference on two-dimensional image recognition, and utilizing the rigid geometric features of the suction nozzle to achieve anti-interference positioning. Furthermore, by extracting feature points from the bag shoulder and constructing a spatial topological constraint model, the initial positioning coordinates of the suction nozzle are verified and corrected using the spatial topological relationship between the shoulder and the suction nozzle. This solves the positioning deviation problem caused by random changes in the posture of flexible bags and the lack of stable correspondence between other parts of the bag and the suction nozzle, avoiding misidentification caused by similar-shaped interference objects or partial occlusion. Finally, by acquiring the suction nozzle position from multiple consecutive frames of images and performing temporal filtering prediction, the stability and continuity of the positioning results are further improved, providing reliable position guidance for high-speed filling operations. Attached Figure Description

[0015] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A flowchart illustrating the steps of the bag opening identification method during beverage bag filling provided in this application. Detailed Implementation

[0016] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] As mentioned in the background section, please refer to the relevant technical issues. Figure 1 This embodiment proposes a bag opening recognition method during beverage bag filling. This method can be applied to automated filling production lines to accurately locate the nozzle position of flexible beverage bags conveyed on a conveying device, and send the positioning result to a filling robot arm or sealing device to achieve accurate insertion and sealing of the filling head. The overall method includes the following steps S100-S400: S100: Acquire an image of the beverage bag on the conveying device, analyze the image, and identify the bag area and the candidate area for the spout on the bag.

[0019] Specifically, in step S100, in practical applications, an industrial camera can be installed above the conveyor device, and with appropriate lighting, image acquisition is triggered when the beverage bag is conveyed to the predetermined shooting position. Since the beverage bag is made of flexible material, its posture may tilt, twist, or wrinkle during conveyance. Directly searching for the nozzle across the entire image would not only be computationally intensive but also easily affected by interference from the conveyor belt background and other equipment components. Therefore, this step first identifies the area where the bag is located, limiting the analysis scope to the inside of the bag to eliminate background interference; then, it further identifies candidate nozzle areas within the bag area. These candidate areas may contain the actual nozzle or other features similar in appearance to the nozzle, such as printed patterns on the bag or protrusions formed by wrinkles, providing an initial search range for subsequent steps, helping to reduce computational load and improve positioning efficiency.

[0020] Furthermore, step S100 specifically includes the following steps S110-S130: S110: Input the bag image into a pre-trained deep learning instance segmentation model to obtain a bag mask and a nozzle candidate area mask.

[0021] Specifically, in step S110, the bag image acquired in step S100 is processed using a pre-trained deep learning instance segmentation model. This model can employ a segmentation network based on a convolutional neural network architecture. In this embodiment, Mask R-CNN or YOLOv8-seg, for example, can be used. It is trained using a large number of training samples labeled with the bag outline and nozzle position, enabling it to simultaneously identify and segment the bag region and the nozzle region.

[0022] After inputting the bag image into the model, the model outputs two masks: a bag mask and a nozzle candidate area mask. The bag mask is a binary image where pixels in the bag area have a value of 1, and pixels in the background area have a value of 0. This mask accurately outlines the overall contour of the bag, providing a basis for subsequent extraction of shoulder feature points. The nozzle candidate area mask is another binary image that identifies areas that may contain the nozzle. Since the nozzle's features are relatively clear in the image, the model can accurately locate its approximate position. However, limited by the two-dimensional image information, this mask may also mark other features with similar appearances on the bag (such as printed patterns, bulges formed by folds, etc.) as candidate areas. Therefore, the nozzle candidate area mask output in this step is a set of candidate areas that includes the actual nozzle, requiring further screening in subsequent steps. By outputting two masks simultaneously through the instance segmentation model, the tediousness of training two independent models is avoided, and the shared features within the model can be utilized to improve detection efficiency and accuracy.

[0023] S120: Extract the outline of the bag body based on the bag body mask, and extract shoulder feature points in the upper half of the bag body outline, the shoulder feature points including left shoulder feature points and right shoulder feature points.

[0024] Specifically, in step S120, after obtaining the bag mask, edge detection is first performed on the mask to extract the bag's outline. In this embodiment, the Canny edge detection algorithm can be used, or the boundary pixels of the mask can be directly extracted to obtain a closed outline curve, which accurately describes the shape and boundary of the bag.

[0025] Because the bag may tilt or twist during transport, its upper structure remains relatively stable. The nozzle is typically located in the center of the upper part of the bag, while the left and right shoulders are located on either side of the upper edge. Therefore, this step involves identifying the left and right shoulder feature points in the upper part of the bag's outline. The extraction of these feature points can be achieved using convex hull analysis: calculating the convex hull of the bag's outline, with the extreme points on either side of the upper half of the convex hull representing the shoulder feature points; alternatively, curvature analysis can be used: calculating the curvature of each point on the outline, with points exhibiting drastic curvature changes typically corresponding to the shoulder positions of the bag. The extracted left and right shoulder feature points serve as key reference points for constructing spatial topological relationships in subsequent steps.

[0026] It should be noted that although the bag is made of flexible material and its overall shape may change during the conveying process, the spatial positional relationship between the left and right shoulder feature points and the nozzle remains relatively stable in a single frame image. This stability provides a feasible basis for the feature points extracted in this step to be used for subsequent positioning.

[0027] S130: Determine the position of the nozzle candidate region based on the nozzle candidate region mask.

[0028] Specifically, in step S130, after obtaining the nozzle candidate area mask, this step converts the mask into specific region location information to guide the subsequent 3D structured light scanning range. Specifically, the minimum bounding rectangle of the nozzle candidate area mask can be calculated; this rectangular region represents the location of the nozzle candidate area. The method for calculating the minimum bounding rectangle includes finding the minimum and maximum abscissas, minimum and maximum ordinates of all non-zero pixels in the mask, thereby determining the boundary of the rectangle.

[0029] Once the location of the nozzle candidate region is determined, the structured light projection unit in the subsequent step S200 will project coded structured light only within this region, rather than scanning the entire bag image. This region-limiting strategy has the following advantages: First, it significantly reduces the computational load of structured light projection and 3D reconstruction, improving the system's processing speed; second, limiting the analysis scope to the candidate region eliminates interference from other parts of the bag, improving the accuracy of nozzle recognition; finally, since the nozzle candidate region is usually small, the structured light projection unit can use higher resolution and denser stripe coding, thereby obtaining higher-precision 3D reconstruction results.

[0030] S200: Obtain the three-dimensional structural information of the nozzle candidate region, and identify the true position of the nozzle based on the three-dimensional structural information to obtain the preliminary positioning coordinates of the nozzle.

[0031] Specifically, in step S200, since the nozzle is made of a rigid material and has a specific three-dimensional geometric shape (such as cylindrical, annular protrusions, etc.), while the beverage bag is made of a flexible material and its surface presents an irregular curved surface in three-dimensional space, this step, by acquiring the three-dimensional structural information of the nozzle candidate region and analyzing the geometric features such as size, shape, and surface normal vector changes in the three-dimensional point cloud data, can accurately identify targets with rigid geometric features. In particular, a natural concave structure is formed at the connection between the nozzle root and the bag body, which can serve as an important basis for distinguishing the nozzle from other protruding features on the bag body. Through this step, even if there is reflection or water mist on the nozzle surface, its three-dimensional geometric structure remains stable and will not change drastically like two-dimensional image features, thus achieving interference-resistant nozzle recognition.

[0032] Furthermore, step S200 specifically includes the following steps S210-S240: S210: Control the structured light projection unit to project coded structured light onto the nozzle candidate area and simultaneously acquire an image with structured light stripes.

[0033] Specifically, in step S210, three-dimensional information of the nozzle candidate area is acquired using structured light three-dimensional measurement technology. The structured light projection unit can be a digital light processing projector or a laser stripe generator, capable of projecting a specifically coded stripe pattern onto the target area. The coding method can employ phase-shifting, Gray code, or multi-frequency heterodyne methods, among which the phase-shifting method can obtain high-precision phase information by projecting multiple phase-shifted sinusoidal stripe images; the Gray code method can achieve large-scale unambiguous absolute phase calculation by projecting alternating black and white coded patterns.

[0034] Since the location of the candidate nozzle area has been determined in step S130, the structured light projection unit can project coded structured light only within this area, instead of scanning the entire bag. This area-limiting strategy not only reduces the workload of the projector but also avoids interference from reflected light from other parts of the bag on the nozzle area. During synchronous acquisition, the industrial camera and the structured light projection unit must maintain strict timing synchronization, that is, the camera exposure is triggered simultaneously with each projection of the stripe pattern to ensure that the acquired image corresponds precisely to the projected stripes. To reduce motion blur, a photoelectric sensor can be installed on the conveyor device to trigger synchronous acquisition when the beverage bag moves to the predetermined position, while simultaneously tracking the bag position in real time using encoder readings.

[0035] S220: Encode and perform phase calculation on the image with structured light stripes to reconstruct the three-dimensional point cloud data of the nozzle candidate region.

[0036] Specifically, in step S220, firstly, each image with structured light stripes is decoded, and the phase value of each pixel is extracted according to the projection encoding method. For example, when using the phase-shifting method, the principal phase value of each pixel is calculated using multiple phase-shifted images; when using the Gray code method, the global encoding is obtained by binarizing the black and white stripe image. The wrapped phase is restored to the absolute phase using a phase unrolling algorithm, thereby determining the projector projection angle corresponding to each pixel.

[0037] After obtaining the absolute phase, the three-dimensional spatial coordinates of each pixel are calculated using triangulation principles, combining pre-calibrated camera intrinsic parameters, projector intrinsic parameters, and the relative pose relationship between the camera and projector. Specifically, for each pixel in the image, its two-dimensional image coordinates are known. The absolute phase determines its corresponding position on the projector's image plane. Then, based on the optical center positions of the camera and projector and the imaging model, the three-dimensional coordinates of that point are calculated using triangulation in stereo vision. After this processing, all pixels within the nozzle candidate region are converted into three-dimensional point cloud data, forming a point cloud set describing the surface geometry of that region. This process elevates two-dimensional image information to three-dimensional space, laying the foundation for subsequent nozzle recognition based on geometric features.

[0038] S230: The three-dimensional point cloud data is filtered, denoised, and clustered to obtain multiple point cloud clusters.

[0039] Specifically, during the 3D point cloud reconstruction process, due to factors such as ambient light interference, reflectivity of the nozzle surface, or complex reflections of the bag surface, the reconstructed point cloud data typically contains noise points and outliers. In step S230, the point cloud is first filtered to improve data quality. Statistical filtering can be used: calculating the average distance between each point and its neighbors; if this distance exceeds a preset statistical threshold, the point is identified as an outlier and removed. Alternatively, radius filtering can be used: if the number of neighbors within a given radius of a point is less than a preset threshold, it is identified as a noise point. After filtering, isolated noise points in the point cloud data are effectively removed.

[0040] After filtering and denoising, this step performs clustering segmentation on the point cloud, dividing the point sets belonging to different objects into independent point cloud clusters. Since the nozzle candidate region may contain the nozzle body, local surfaces of the bag, and other attachments, it is necessary to distinguish them. Clustering algorithms can include Euclidean clustering: setting a distance threshold to group spatially close points into the same cluster; alternatively, a region growing algorithm based on normal vector differences can be used to merge regions with similar geometric features into one cluster. After clustering segmentation, the originally continuous point cloud is divided into multiple independent point cloud clusters, each corresponding to a potential object candidate.

[0041] S240: Perform geometric feature analysis on each point cloud cluster. The geometric features include size, shape, surface normal vector variation, and the degree of concavity at the connection between the bottom and the surrounding area. Point cloud clusters that meet the rigid geometric features of the nozzle and whose concavity exceeds a preset concavity threshold are determined to be the real nozzles. The geometric center of the point cloud cluster is used as the initial positioning coordinates of the nozzle.

[0042] Specifically, in step S240, firstly, dimensional characteristics are analyzed: based on the actual physical dimensions of the nozzle, a reasonable range for length, width, and height is preset, and point cloud clusters whose dimensions exceed this range are excluded. Secondly, shape characteristics are analyzed: nozzles are usually cylindrical or have a regular annular structure. Whether they conform to the geometric shape of a rigid nozzle can be determined by calculating the principal curvature of the point cloud clusters, fitting a cylinder, and calculating the fitting error. Thirdly, surface normal vector changes are analyzed: the nozzle is made of rigid material with a smooth surface, resulting in a gradual change in normal vector; while the bag body is made of flexible material, and the surface may have wrinkles, leading to drastic changes in normal vector. The standard deviation of the normal vector can be statistically analyzed to effectively distinguish between the two.

[0043] Specifically, this step focuses on analyzing the degree of indentation at the connection between the bottom of the point cloud cluster and the surrounding area. Due to the flexible nature of the bag, a recessed area naturally forms at the connection between the nozzle and the bag body. This indentation is a key feature that distinguishes the nozzle from other protruding features on the bag body. The degree of indentation can be calculated by analyzing the height difference between the bottom edge of the point cloud cluster and the surrounding bag point cloud: extract the bottom edge points of the point cloud cluster and calculate the vertical distance between these points and the adjacent bag point cloud; the greater the distance, the deeper the indentation. Alternatively, the rate of curvature change at the connection can be calculated; the curvature change in the recessed area is significantly greater than that on a flat surface. The degree of indentation is compared with a preset indentation threshold. Only point cloud clusters that simultaneously meet the rigid geometric characteristics and have an indentation degree exceeding the threshold are considered true nozzles.

[0044] S300: Extract the shoulder feature points of the bag area, construct a dynamic constraint model based on the spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the nozzle, and use the dynamic constraint model to verify and correct the initial positioning coordinates of the nozzle to obtain the corrected positioning coordinates of the nozzle.

[0045] Specifically, in step S300, although the beverage bag is made of flexible material and its posture may change in various ways, there is a relatively stable spatial relationship between the shoulder area on the upper part of the bag and the spout. For example, the spout is usually located in the center of the upper part of the bag, and the distance and angle between it and the feature points on the left and right shoulders remain relatively stable within a certain range. This step constructs a dynamic constraint model describing the spatial relationship between the spout and the shoulder by extracting the feature points of the left and right shoulders of the upper half of the bag's outline. This model can learn and update the expected value of the geometric relationship between the shoulder and the spout online, and verify and correct the initial positioning coordinates of the spout based on the deviation between the current measurement value and the expected value, thereby effectively avoiding positioning errors caused by local occlusion, similar-shaped interference objects, or spout detection failure.

[0046] Furthermore, step S300 specifically includes the following steps S310-S350: S310: Convert the two-dimensional image coordinates of the left shoulder feature point and the right shoulder feature point into estimated three-dimensional spatial coordinates.

[0047] Specifically, because the bag body is made of flexible material and the shoulder area typically lacks the rigid geometric features of a suction nozzle, it is difficult to obtain accurate 3D information through structured light reconstruction. Therefore, this step uses an estimation method to obtain the 3D coordinates of the shoulder feature points.

[0048] In step S310, the depth value of the shoulder feature point is estimated using the 3D point cloud data reconstructed in step S220, combined with the positions of the left and right shoulder feature points in the image, through interpolation or neighborhood search methods. For each shoulder feature point, effective nearby points are found in the 3D point cloud centered on its 2D image coordinates, and the average depth of these points is taken as the depth value of the shoulder feature point. Since the shoulder area is located at the top of the bag, its depth change is relatively gradual, and this estimation method can obtain reasonable approximate 3D coordinates. When the shoulder area is not covered by the nozzle candidate area, the overall planar assumption of the bag area can also be used to estimate the depth information of the shoulder in combination with camera calibration parameters.

[0049] S320: Calculate the first spatial distance between the initial positioning coordinates of the suction nozzle and the feature point of the left shoulder, the second spatial distance between the nozzle and the feature point of the right shoulder, and the spatial angle between the first vector pointing from the suction nozzle to the feature point of the left shoulder and the second vector pointing from the suction nozzle to the feature point of the right shoulder.

[0050] Specifically, in step S320, the first spatial distance and the second spatial distance are calculated using the Euclidean distance formula: the straight-line distance between the initial positioning coordinates of the nozzle and the feature point on the left shoulder, and the straight-line distance between the nozzle and the feature point on the right shoulder are calculated respectively. These two distances reflect the lateral position of the nozzle on the bag body. When the nozzle is located in the center of the bag body, the two distances are basically equal; when the bag body tilts or twists, the two distances will change accordingly.

[0051] The calculation process for the spatial angle is as follows: First, construct a first vector pointing from the nozzle to the left shoulder feature point, and a second vector pointing from the nozzle to the right shoulder feature point; then calculate the angle between these two vectors. This angle reflects the opening angle of the nozzle relative to the shoulder and is closely related to the deformation state of the bag. When the bag is flat and unfolded, the angle is larger; when the bag is wrinkled or bent, the angle will decrease accordingly. Through these three geometric parameters, this step quantifies the spatial positional relationship between the nozzle and the shoulder into specific numerical values, providing measurable input for the subsequent dynamic constraint model.

[0052] S330: The first spatial distance, the second spatial distance, and the spatial angle are used as the current measured values ​​and compared with the expected values ​​maintained by the dynamic constraint model to calculate the degree of deviation between the current measured values ​​and the expected values.

[0053] Specifically, the core of step S330 is the dynamic constraint model, which maintains the expected value of the spatial topological relationship between the nozzle and the shoulder. During system initialization, the expected value can be obtained by acquiring several frames of bag images in normal posture and calculating the statistical mean of the first spatial distance, the second spatial distance, and the spatial angle. During system operation, the expected value is continuously adjusted via online updates in step S350 to adapt to differences caused by different bag batches or posture variations.

[0054] The deviation is calculated using Mahalanobis distance or weighted Euclidean distance. Specifically, for the current frame's measured value vector and the expected value vector, the difference between the two is calculated, and then weighted and normalized according to the variance of each parameter to obtain the comprehensive deviation. This weighting method considers the fluctuation range of different geometric parameters under normal conditions, allowing the deviation to objectively reflect the degree of abnormality of the current measured value. When the initial positioning coordinates of the nozzle are accurate, the current measured value is close to the expected value, and the deviation is small; when the initial positioning coordinates of the nozzle deviate from the true position due to misidentification, the current measured value differs significantly from the expected value, and the deviation is large.

[0055] S340: Verify the nozzle's corrected positioning coordinates based on the comparison result between the deviation degree and the preset deviation threshold.

[0056] Specifically, in step S340, the preset deviation threshold can be set according to the actual application scenario, and usually includes one or more thresholds to distinguish different levels of deviation. When the deviation is small, it indicates that the current initial positioning coordinates of the nozzle are in good agreement with the expected value of the dynamic constraint model, and the coordinates are reliable. When the deviation increases, it indicates that the initial positioning coordinates of the nozzle may have some error and need to be corrected by combining the model prediction value. When the deviation is too large, it indicates that the initial positioning coordinates of the nozzle are likely to be a misidentification result and should be discarded and replaced by the model prediction value.

[0057] Furthermore, step S340 specifically includes the following steps S341-S343. It should be noted that steps S341-S343 are executed in parallel, and their sequence numbers are only for illustrative purposes and are not intended to limit their sequential execution.

[0058] S341: If the deviation is less than the first preset deviation threshold, the initial positioning coordinates of the nozzle are directly used as the corrected positioning coordinates of the nozzle.

[0059] Specifically, in step S341, the first preset deviation threshold is a low threshold set by the dynamic constraint model to determine the degree of agreement between the current measurement value and the expected value. When the deviation is less than this threshold, it indicates that the initial positioning coordinates of the nozzle in the current frame are highly consistent with the expected value maintained by the dynamic constraint model, meaning that the spatial topological relationship between the nozzle and the shoulder is completely within the normal fluctuation range. This situation usually occurs when the nozzle is accurately identified, the bag posture is stable, and there is no obvious interference. In this case, step S340 directly uses the initial positioning coordinates of the nozzle as the corrected positioning coordinates, without the need for additional correction.

[0060] S342: If the deviation is greater than or equal to the first preset deviation threshold and less than the second preset deviation threshold, then the preliminary positioning coordinates of the nozzle and the predicted position predicted by the dynamic constraint model are weighted and fused to obtain the corrected positioning coordinates of the nozzle.

[0061] Specifically, the second preset deviation threshold is a higher threshold set by the dynamic constraint model to determine whether the measured value is severely abnormal. In step S342, when the deviation is between the first and second preset deviation thresholds, it indicates that the current initial positioning coordinates of the nozzle deviate from the expected value to a certain extent, but have not yet reached the level of severe abnormality. This situation may be caused by the following reasons: the bag posture has changed within the normal range, slight reflection on the nozzle surface causes a slight shift in positioning, or slight vibration of the conveying device, etc.

[0062] To address this issue, this embodiment employs a weighted fusion approach, comprehensively considering both the initial positioning coordinates of the suction nozzle and the predicted position from the dynamic constraint model. The core idea of ​​weighted fusion is to dynamically adjust the weights based on the degree of deviation; the greater the deviation, the smaller the weight of the initial positioning coordinates and the larger the weight of the model-predicted position. Specifically, the fusion weights can be set to be inversely proportional to the degree of deviation, or a linear interpolation method can be used to calculate the fusion result. Through weighted fusion, effective information reflecting the actual position in the initial positioning coordinates is retained, while the model's predicted values ​​compensate for potential measurement errors, resulting in smoother and more stable corrected positioning coordinates. This weighted fusion mechanism effectively suppresses positioning fluctuations caused by minor interference.

[0063] S343: If the deviation is greater than or equal to the second preset deviation threshold, the initial positioning coordinates of the nozzle are discarded, and the predicted position predicted by the dynamic constraint model is used as the corrected positioning coordinates of the nozzle.

[0064] Specifically, in step S343, when the deviation exceeds the threshold, it indicates that the deviation between the current initial positioning coordinates of the nozzle and the expected value has far exceeded the normal fluctuation range. This usually means that the initial positioning coordinates of the nozzle are a misidentification result, rather than the actual nozzle position. Common misidentification scenarios include: misidentifying a printed pattern on the bag as a nozzle, misidentifying a reflective point as a nozzle, or the algorithm outputting an incorrect result when the nozzle cannot be properly imaged due to obstruction.

[0065] To address this issue, this embodiment directly discards the initial positioning coordinates of the nozzle and instead uses the predicted position from the dynamic constraint model as the corrected positioning coordinates for the nozzle. The model-predicted position is calculated based on the nozzle movement trends and spatial topological relationships of historical frames. Even if nozzle detection fails in the current frame, the predicted position can still accurately reflect the actual position of the nozzle. This processing mechanism allows the system to output reliable positioning results even when facing interference such as partial occlusion and strong reflections, avoiding filling interruptions caused by single-frame detection failures.

[0066] S350: Update the expected value and variance of the dynamic constraint model online using the validated current measurements.

[0067] Specifically, in step S350, the verified measurement values ​​include the first spatial distance, the second spatial distance, and the spatial angle. These values ​​have been verified in step S340 and are considered to be the true spatial topological relationship between the nozzle and the shoulder in the current frame.

[0068] The expected value can be updated using an exponentially weighted moving average method: the new expected value equals the current measurement multiplied by the updated weights, plus the original expected value multiplied by the weight decay factor, allowing the model to gradually adapt to the slow changes in the bag's posture. The variance update reflects the degree of fluctuation in the measurements: when measurements are stable across multiple frames, the variance decreases, increasing the model's sensitivity to deviations; when measurements fluctuate significantly, the variance increases, increasing the model's tolerance to deviations. This online update mechanism makes the dynamically constrained model adaptive, enabling it to track changes in the bag's posture under different working conditions, avoiding the poor adaptability of fixed templates in practical applications.

[0069] S400: Collect the corrected positioning coordinates of the same nozzle in a predetermined number of consecutive frames of images, form the nozzle motion trajectory, and predict the position of the nozzle motion trajectory based on the time series filtering algorithm, and output the final positioning coordinates of the nozzle.

[0070] Specifically, in step S400, in an actual production line, the beverage bag moves at a certain speed with the conveyor, and the movement trajectory of the nozzle should remain smooth and continuous for a short period of time without abrupt changes. However, due to factors such as image acquisition noise and algorithm errors, the positioning result of a single frame may exhibit random fluctuations. This step acquires the corrected positioning coordinates of the same nozzle in multiple consecutive frames of images to form a movement trajectory, and uses a time series filtering algorithm (such as Kalman filtering) to smooth the trajectory, which can effectively suppress random noise and improve positioning accuracy. In addition, this step also uses a filtering model to predict the position of the nozzle in the next frame of the image, and sends the prediction result to the actuator in advance to achieve look-ahead control, compensate for mechanical response delay, and ensure positioning accuracy during high-speed filling.

[0071] Furthermore, step S400 specifically includes the following steps S410-S440: S410: Construct a state vector and a state transition model, wherein the state vector includes the position component and velocity component of the nozzle, and the state transition model is used to describe the change law of the state vector.

[0072] Specifically, in practical applications, because the beverage bag moves at a constant speed with the conveyor, the position change of the nozzle in multiple consecutive frames of images is regular. In step S410, the motion process of the nozzle is modeled as a dynamic system by constructing a state vector and a state transition model. The state vector is a set of variables describing the system state. In this step, it is designed to include the three-dimensional position component and the three-dimensional velocity component of the nozzle, for a total of six dimensions. The position component represents the coordinates of the nozzle in space, and the velocity component represents the velocity of the nozzle in three directions.

[0073] The state transition model describes how the state vector changes over time. Under the assumption of uniform motion, the position at the next moment is equal to the current position plus the velocity multiplied by the time interval, while the velocity remains constant. Based on this principle, a linear state transition matrix can be constructed to map the state vector of the previous frame to the predicted state of the current frame. Since slight velocity fluctuations or vibrations may occur during actual transmission, the state transition model also includes a process noise covariance matrix to describe the uncertainty of the model's predictions.

[0074] S420: Based on the posterior state estimate of the previous frame and the state transition model, predict the prior state estimate of the current frame.

[0075] Specifically, in step S420, the posterior state estimate of the previous frame is the optimal estimate after filtering and fusion, which includes the nozzle position and velocity information of the previous frame. Substituting the posterior state estimate into the state transition model yields the prior state estimate of the current frame.

[0076] The physical meaning of prior state estimation lies in inferring the current position of the nozzle based solely on historical information without observing the current frame image. This prediction fully utilizes the continuity of motion, ensuring a reliable position reference even if there is delay or noise in the nozzle detection results of the current frame. Simultaneously, this step also calculates the covariance matrix of the prior state estimation, which describes the degree of uncertainty in the prediction result. The uncertainty of the prior estimation is small when the conveyor operates smoothly; however, the uncertainty increases accordingly when there are speed fluctuations or vibrations.

[0077] S430: The corrected positioning coordinates of the nozzle in the current frame are fused with the prior state estimate as the observation value to obtain the posterior state estimate of the current frame, and the position component is extracted from the posterior state estimate as the final positioning coordinates of the nozzle.

[0078] Specifically, in step S430, the fusion process is based on the Kalman filter framework: first, the Kalman gain is calculated, which reflects the relative confidence between the observation and the prior estimate; then, the prior estimate is corrected based on the difference between the observation and the prior estimate to obtain the posterior state estimate.

[0079] The goal of fusion is to minimize the variance of the estimation error, making the posterior state estimate statistically superior to the prior estimate and the observed values. When the noise in the observed values ​​is low, the Kalman gain is high, and the fusion result is more inclined towards the observed values; when the confidence level of the prior estimate is high, the fusion result is more inclined towards the prior estimate. This adaptive fusion effectively suppresses random fluctuations in single-frame localization results, improving the stability of localization. After fusion, the position component is extracted from the posterior state estimate as the final localization coordinates of the nozzle. These coordinates integrate historical motion trends and current observation information, maintaining the smoothness of the motion trajectory while reflecting the actual position changes of the nozzle in a timely manner.

[0080] S440: Predict the position of the nozzle in the next frame based on the posterior state estimation, and send the prediction result to the actuator for look-ahead control.

[0081] Specifically, in step S440, the prediction process is similar to that in step S420, that is, the posterior state estimate of the current frame is substituted into the state transition model to calculate the prior state estimate of the next frame, and the position component is extracted from it as the predicted value of the nozzle position in the next frame.

[0082] In actual filling processes, the filling robot arm or sealing device experiences mechanical response delays, requiring a certain amount of time from receiving the positioning command to completing the action. If only the positioning result of the current frame is used for control, the nozzle may have already moved a distance with the conveyor when the actuator reaches the designated position, leading to alignment deviations. By predicting the nozzle position in the next frame and sending it to the actuator in advance, the system delay can be compensated for, achieving synchronous following between the filling head and the nozzle. This look-ahead control mechanism makes this application applicable to high-speed filling production lines, ensuring positioning accuracy without affecting production efficiency.

[0083] Furthermore, the execution of step S100 also includes steps S140-S160. It should be noted that the sequence numbers of steps S140-S160 are used for ease of explanation and are not intended to be located after step S130.

[0084] S140: If the candidate area of ​​the suction nozzle cannot be identified, re-acquire the bag image and re-identify it.

[0085] Specifically, during the analysis of the bag image in step S100, the candidate area for the suction nozzle may fail to be identified due to various reasons, such as sudden changes in lighting causing the image to be too dark or overexposed, the bag position deviating from the center of the camera's field of view, or motion blur caused by vibration of the conveyor device. To address these occasional problems, an immediate retry mechanism has been designed.

[0086] When the deep learning instance segmentation model fails to output a nozzle candidate region mask, or the output mask area is too small, the system determines that it cannot identify the nozzle candidate region. In this case, the camera is controlled to re-acquire the bag image, and the image analysis operation in step S100 is re-executed. During the retry, the camera exposure parameters can be adjusted appropriately to adapt to changes in lighting, or the image acquisition timing can be fine-tuned based on encoder readings to improve the retry success rate. The retry mechanism is designed based on the following considerations: occasional interference can often be eliminated in a short time; immediate retry can restore the recognition function at a lower cost, avoiding triggering more complex error handling processes due to a single frame failure.

[0087] S150: If the retry fails, the nozzle position of the next frame predicted by the time series filtering algorithm will be used as the final positioning coordinates of the current frame.

[0088] Specifically, in step S150, if the retry operation still fails to identify the nozzle candidate region, it indicates that no valid observation value can be obtained in the current frame. At this time, this step uses the nozzle position predicted by the time series filtering algorithm in the next frame as the final positioning coordinates of the current frame. This prediction value is based on historical motion trajectory calculation and can accurately estimate the actual position of the nozzle in a short time, allowing the filling operation to continue and avoiding downtime due to single-frame detection failure.

[0089] S160: When the candidate area of ​​the nozzle cannot be identified for three consecutive frames, the detection threshold of the deep learning instance segmentation model is reduced or the backup detection algorithm is switched. If it still fails, an alarm signal is issued and the filling operation of the current beverage bag is suspended.

[0090] Specifically, in step S160, if no detection is found for three consecutive frames, it indicates persistent interference. This step first lowers the detection threshold to include candidate regions that were previously filtered out due to slightly low confidence. If this is still ineffective, a backup detection algorithm (such as template matching) is switched to. If all the above strategies fail, it is determined that the current beverage bag cannot be filled normally, the system issues an alarm signal, suspends the filling operation of the bag, and marks it as an abnormal product.

[0091] Furthermore, in step S200, the process of obtaining the three-dimensional structural information of the nozzle candidate region and identifying the true position of the nozzle based on the three-dimensional structural information also includes steps S250-S260. It should be noted that the sequence numbers of steps S250-S260 are only for ease of explanation and are not limited to being located after step S240.

[0092] S250: If the actual nozzle cannot be identified from the 3D point cloud data, adjust the structured light projection intensity, re-acquire an image with structured light stripes, and reconstruct the 3D point cloud data.

[0093] Specifically, during the process of acquiring the three-dimensional structural information of the nozzle candidate region in step S200, the quality of the reconstructed three-dimensional point cloud may be poor due to reasons such as excessively strong or weak reflection on the nozzle surface or water mist obscuring the surface of the bag, making it impossible to identify point cloud clusters that meet the rigid geometric characteristics of the nozzle. When all point cloud clusters in step S240 do not meet the nozzle determination conditions, this step activates the error handling mechanism.

[0094] First, the projection intensity of the structured light projection unit is adjusted. For cases of excessive reflection, the projection intensity can be reduced to decrease overexposed areas caused by specular reflection; for cases where the nozzle surface has high absorptivity resulting in weak signals, the projection intensity can be increased to enhance stripe contrast. After adjusting the intensity, an image with structured light stripes is reacquired, and the 3D point cloud reconstruction operation is re-performed. This retry mechanism utilizes the adaptability of different projection intensities to different surface materials, which can improve point cloud quality and increase the nozzle recognition success rate in a short time. If the actual nozzle still cannot be recognized after retrying, the process proceeds to the next step of degradation processing.

[0095] S260: If the retry fails, the three-dimensional position of the nozzle is estimated based on the two-dimensional image coordinates of the nozzle candidate area combined with the historical average depth data of the bag area, and the estimated position coordinates are used for subsequent verification and correction.

[0096] Specifically, in step S260, if the actual nozzle cannot be identified from the 3D point cloud even after adjusting the structured light projection intensity, it indicates that the 3D reconstruction can no longer obtain effective information in the current frame. At this time, this step adopts a degradation strategy, using 2D image information combined with historical depth data to estimate the 3D position of the nozzle.

[0097] Specifically, based on the candidate nozzle region determined in step S130, the center point of the two-dimensional image coordinates of that region is calculated. Simultaneously, based on the successfully reconstructed nozzle depth data from historical frames, the average depth value of the bag region is calculated. Since the nozzle is usually located on the bag surface, its depth is close to the average depth of the bag, and this average depth value can be used as an estimate of the nozzle's depth in the current frame. The center point of the two-dimensional image coordinates is combined with the estimated depth value, and the estimated three-dimensional position coordinates are obtained through backprojection using camera intrinsic parameters. These estimated coordinates serve as the initial positioning coordinates of the nozzle and are sent to step S300 for subsequent verification and correction processing.

[0098] Furthermore, in step S300, the process of constructing a dynamic constraint model based on the spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the nozzle, and using the dynamic constraint model to verify and correct the initial positioning coordinates of the nozzle, also includes step S360. It should be noted that the sequence number of step S360 is only for ease of explanation and is not limited to its location after step S350.

[0099] S360: If an abnormal situation occurs where the deviation is greater than or equal to the second preset deviation threshold in three consecutive frames, the expected value and variance of the dynamic constraint model are reinitialized, and the operator is prompted to confirm whether the bag type or the status of the conveying device has changed.

[0100] Specifically, in step S300, during the process of constructing a dynamic constraint model based on the shoulder feature points and the initial positioning coordinates of the nozzle and performing verification and correction, although step S343 has adopted the method of removing measured values ​​and using model prediction values ​​for serious deviations in a single frame, if multiple consecutive frames show an abnormal situation where the degree of deviation is greater than or equal to the second preset deviation threshold, it indicates that the problem may not be an occasional misidentification, but rather a continuous change in the system state.

[0101] When the above anomalies occur in three consecutive frames, this step determines that the expected value and variance currently maintained by the dynamic constraint model can no longer accurately describe the spatial topological relationship between the nozzle and the shoulder. This situation is usually caused by the following reasons: the production line has changed to different specifications of beverage bags, causing a fundamental change in the geometric relationship between the nozzle and the shoulder; the conveyor device has shifted or its speed has changed, altering the movement trajectory of the nozzle; or the camera mounting position has become loose, causing the coordinate transformation parameters to fail. For these problems, continuing to use the original expected value and variance for verification will be meaningless and may even exacerbate misjudgments. Therefore, this step reinitializes the expected value and variance of the dynamic constraint model. The initialization operation can set the expected value to the statistical mean of the measured values ​​in the current consecutive abnormal frames, and set the variance to the initial preset value or calculate it based on the fluctuation of the measured values. Through reinitialization, the dynamic constraint model is reset, establishing a new reference benchmark for verification of subsequent frames.

[0102] Furthermore, step S400 also includes the following steps S450-S460. It should be noted that the sequence numbers of steps S450-S460 are only for ease of explanation and are not intended to be located after step S440.

[0103] S450: Calculate the difference between the current frame observation and the prior state estimate.

[0104] Specifically, in step S450, the current frame observation is the nozzle correction positioning coordinate used for fusion in step S430, and the prior state estimate is the current frame nozzle position predicted based on historical information in step S420. This step calculates the difference between the two, which reflects the degree of deviation between the observed value and the predicted value.

[0105] During normal operation of the filtering system, the difference should remain within a small range, conforming to the system's noise statistics. An abnormally large difference indicates a significant discrepancy between the observed and predicted values, which may be caused by: a large error in nozzle detection, a sudden change in the conveyor speed, or a drastic change in bag posture. Calculating the difference provides a quantitative basis for subsequent anomaly assessment.

[0106] S460: If the difference exceeds a preset threshold for three consecutive frames, the process noise covariance of the state transition model is increased, and the filling speed is reduced or an alarm signal is issued.

[0107] Specifically, when the difference calculated in step S450 exceeds the preset threshold for three consecutive frames, it indicates that the filtering system has a continuous inconsistency between observation and prediction, rather than random fluctuations in a single frame.

[0108] In step S460, when the difference calculated in step S450 exceeds a preset threshold for three consecutive frames, it indicates that the filtering system is experiencing a persistent inconsistency between observation and prediction. At this point, this step performs two adjustments: First, it increases the process noise covariance of the state transition model, causing the Kalman filter to reduce its confidence in the model's predictions and increase its confidence in the observations during the fusion process, thus adapting to conditions such as changes in the conveyor speed or sudden changes in bag posture. Second, it reduces the filling speed or issues an alarm signal. Reducing the speed provides the system with more processing time, while the alarm notifies the operator to check the conveyor or nozzle detection for any abnormalities. Through this tiered response mechanism, this step effectively controls the filling quality risks that may be caused by abnormal conditions while maintaining continuous system operation.

[0109] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for identifying the bag opening during beverage bag filling, characterized in that, include: Acquire an image of the beverage bag on the conveying device, and analyze the image to identify the bag area and the candidate area for the spout on the bag. The three-dimensional structural information of the candidate region of the suction nozzle is obtained, and the actual position of the suction nozzle is identified based on the three-dimensional structural information to obtain the preliminary positioning coordinates of the suction nozzle; Extract shoulder feature points from the bag area, construct a dynamic constraint model based on the spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the nozzle, and use the dynamic constraint model to verify and correct the initial positioning coordinates of the nozzle to obtain the corrected positioning coordinates of the nozzle. The corrected positioning coordinates of the same nozzle are acquired in a predetermined number of consecutive frames of images to form the nozzle motion trajectory. The position of the nozzle motion trajectory is predicted based on a time series filtering algorithm, and the final positioning coordinates of the nozzle are output.

2. The method for identifying the bag opening during beverage bag filling according to claim 1, characterized in that, The analysis of the bag image to identify the bag area and candidate areas for the suction nozzle on the bag includes: The bag image is input into a pre-trained deep learning instance segmentation model to obtain a bag mask and a nozzle candidate region mask. The bag outline is extracted based on the bag mask, and shoulder feature points are extracted in the upper half of the bag outline, including left shoulder feature points and right shoulder feature points; The position of the nozzle candidate region is determined based on the nozzle candidate region mask.

3. The method for identifying the bag opening during beverage bag filling according to claim 1, characterized in that, The step of obtaining the three-dimensional structural information of the candidate region of the suction nozzle and identifying the true position of the suction nozzle based on the three-dimensional structural information includes: The structured light projection unit is controlled to project coded structured light onto the candidate area of ​​the suction nozzle, and images with structured light stripes are acquired simultaneously. The image with structured light stripes is encoded and phase calculated to reconstruct the three-dimensional point cloud data of the nozzle candidate region; The three-dimensional point cloud data is filtered, denoised, and clustered to obtain multiple point cloud clusters; Geometric feature analysis is performed on each point cloud cluster. The geometric features include size, shape, surface normal vector variation, and the degree of concavity at the connection between the bottom and the surrounding area. Point cloud clusters that meet the rigid geometric features of the nozzle and whose concavity exceeds a preset concavity threshold are identified as real nozzles. The geometric center of the point cloud cluster is used as the initial positioning coordinates of the nozzle.

4. The method for identifying the bag opening during beverage bag filling according to claim 1, characterized in that, The process of constructing a dynamic constraint model based on the spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the suction nozzle, and using the dynamic constraint model to verify and correct the initial positioning coordinates of the suction nozzle, includes: The two-dimensional image coordinates of the left and right shoulder feature points are converted into estimated three-dimensional spatial coordinates. Calculate the first spatial distance between the initial positioning coordinates of the suction nozzle and the feature point of the left shoulder, the second spatial distance between the nozzle and the feature point of the right shoulder, and the spatial angle between the first vector pointing from the suction nozzle to the feature point of the left shoulder and the second vector pointing from the suction nozzle to the feature point of the right shoulder; The first spatial distance, the second spatial distance, and the spatial angle are used as the current measured values ​​and compared with the expected values ​​maintained by the dynamic constraint model to calculate the degree of deviation between the current measured values ​​and the expected values. The correction positioning coordinates of the nozzle are determined by verifying the deviation degree against the preset deviation threshold. The expected value and variance of the dynamic constraint model are updated online using the validated current measurements.

5. The method for identifying the bag opening during beverage bag filling according to claim 4, characterized in that, The step of verifying the nozzle's corrected positioning coordinates based on a comparison between the deviation degree and a preset deviation threshold specifically includes: If the deviation is less than the first preset deviation threshold, the initial positioning coordinates of the nozzle are directly used as the corrected positioning coordinates of the nozzle. If the deviation is greater than or equal to the first preset deviation threshold and less than the second preset deviation threshold, then the initial positioning coordinates of the nozzle and the predicted position predicted by the dynamic constraint model are weighted and fused to obtain the corrected positioning coordinates of the nozzle. If the deviation is greater than or equal to the second preset deviation threshold, the initial positioning coordinates of the nozzle are discarded, and the predicted position predicted by the dynamic constraint model is used as the corrected positioning coordinates of the nozzle.

6. The method for identifying the bag opening during beverage bag filling according to claim 1, characterized in that, The corrected positioning coordinates of the same suction nozzle are acquired in a predetermined number of consecutive image frames to form the nozzle's motion trajectory. The position of the suction nozzle's motion trajectory is then predicted based on a time-series filtering algorithm, including: Construct a state vector and a state transition model. The state vector includes the position component and velocity component of the nozzle. The state transition model is used to describe the change law of the state vector. Based on the posterior state estimate of the previous frame and the state transition model, the prior state estimate of the current frame is predicted. The nozzle correction positioning coordinates of the current frame are fused with the prior state estimate as the observation value to obtain the posterior state estimate of the current frame, and the position component is extracted from the posterior state estimate as the final positioning coordinates of the nozzle. The position of the nozzle in the next frame is predicted based on the posterior state estimation, and the prediction result is sent to the actuator for look-ahead control.

7. The method for identifying the bag opening during beverage bag filling according to claim 2, characterized in that, The process of analyzing the bag image also includes: If the candidate area for the suction nozzle cannot be identified, the bag image is re-acquired and re-identified. If the retry fails, the nozzle position of the next frame predicted by the time series filtering algorithm will be used as the final positioning coordinates of the current frame. If the candidate region for the sip nozzle cannot be identified for three consecutive frames, the detection threshold of the deep learning instance segmentation model is lowered or the backup detection algorithm is switched. If the detection still fails, an alarm signal is issued and the filling operation of the current beverage bag is paused.

8. The method for identifying the bag opening during beverage bag filling according to claim 3, characterized in that, The process of obtaining the three-dimensional structural information of the candidate region of the suction nozzle and identifying the true position of the suction nozzle based on the three-dimensional structural information also includes: If the actual nozzle cannot be identified from the 3D point cloud data, adjust the structured light projection intensity, re-acquire images with structured light stripes, and reconstruct the 3D point cloud data. If the retry fails, the three-dimensional position of the nozzle is estimated based on the two-dimensional image coordinates of the nozzle candidate area and the historical average depth data of the bag area, and the estimated position coordinates are used for subsequent verification and correction.

9. The method for identifying the bag opening during beverage bag filling according to claim 4, characterized in that, The process of constructing a dynamic constraint model based on the spatial topological relationship between the shoulder feature points and the initial positioning coordinates of the suction nozzle, and using the dynamic constraint model to verify and correct the initial positioning coordinates of the suction nozzle, also includes: If an abnormal situation is detected in which three consecutive frames show a deviation greater than or equal to the second preset deviation threshold, the expected value and variance of the dynamic constraint model are reinitialized, and the operator is prompted to confirm whether the bag type or the status of the conveying device has changed.

10. The method for identifying the bag opening during beverage bag filling according to claim 6, characterized in that, Also includes: Calculate the difference between the current frame observation and the prior state estimate; If the difference exceeds a preset threshold for three consecutive frames, the process noise covariance of the state transition model is increased, and the filling speed is reduced or an alarm signal is issued.

Images