Disposable liquid preparation system automatic checking and error prevention method based on AI visual recognition

Abstract

The present application relates to the technical field of disposable liquid preparation system assembly detection, and particularly relates to an AI vision-based automatic checking and error prevention method for disposable liquid preparation system, which comprises the following steps: determining component recognition results; determining actual assembly relationships corresponding to each conveying pipeline; determining connection sequence information of each conveying pipeline; determining abnormal assembly positions and types; and generating error prevention control instructions. The present application firstly recovers actual connection relationships after assembly completion based on multi-angle vision acquisition results, converts the actual connection relationships into connection sequence information reflecting liquid conveying paths, and compares the connection sequence information with standard connection sequences in corresponding liquid preparation process stages item by item, so as to not only identify local assembly abnormalities such as missing components and connection object errors, but also effectively solve the problem that pipeline misconnection and reverse connection and other assembly errors are difficult to be automatically intercepted before operation due to lack of automatic recognition capability for the overall connection state of the system.

Description

Technical Field

[0001] This invention relates to the field of assembly and testing technology for disposable liquid dispensing systems, and in particular to an automatic verification and error prevention method for disposable liquid dispensing systems based on AI visual recognition. Background Technology

[0002] Disposable liquid preparation systems consist of single-use components such as preparation bags, delivery pipelines, filters, sterile connectors, and online sensors. Due to their advantages of requiring no cleaning or sterilization, low risk of cross-contamination, and flexible production changeover, they are widely used in biopharmaceuticals, vaccine production, antibody drugs, and fermentation engineering. In actual production, disposable liquid preparation systems typically require the assembly of multiple delivery pipelines with preparation bags, filters, sterile connectors, and other functional components according to specified connection relationships, based on different liquid preparation process requirements. Furthermore, different pipelines must not only ensure complete interface connections but also meet the requirements for liquid delivery direction and process flow sequence. However, due to the large number of delivery pipelines in disposable liquid preparation systems, and the flexible, transparent, or semi-transparent nature of these pipelines, issues such as pipeline crossings, obstructions, and localized morphological changes can easily occur after assembly. This makes it difficult for manual personnel to accurately trace the complete connection path of each pipeline and verify the connection objects at both ends and the liquid flow direction. Currently, the industry mainly relies on manual visual inspection and assembly record verification for verification, which cannot achieve automatic confirmation of the overall assembly topology.

[0003] While existing visual inspection technologies can identify individual components or local interfaces, in the assembly scenario of a disposable liquid preparation system, the low contrast between transparent tubing and the background, coupled with the bending, overlapping, and obstruction of flexible tubing, makes it difficult for conventional visual recognition methods to reliably capture the complete pipeline route and its spatial connections. This makes it impossible to further determine whether multiple components form an actual transport path that meets process requirements. Therefore, when assembly anomalies such as incorrect, missing, or reversed pipe connections occur, existing detection methods often struggle to accurately identify them before liquid preparation begins, potentially leading to erroneous assembly entering the production process and causing batches of high-value materials to be scrapped.

[0004] The existing technology has at least the following problems: First, the existing assembly verification methods for disposable liquid preparation systems mainly rely on manual inspection of assembly drawings or process records, which cannot automatically obtain the actual connection relationship between the components after assembly, making it difficult to detect abnormalities such as incorrect or missing pipe connections in a timely manner. Second, existing vision-based detection methods usually identify individual components or local connection positions. Due to the imaging characteristics of transparent flexible pipes and the influence of multiple pipe intersections and obstructions, it is difficult to accurately establish the overall connection relationship between the delivery pipes and each component, and it is impossible to determine whether the actual liquid delivery path meets the preset process requirements. Third, the existing technology lacks an overall flow path verification mechanism for the assembly state of the disposable liquid preparation system, and cannot identify assembly abnormalities such as reversed pipe connections and incorrect connection sequences that affect the process before liquid preparation operation, which may lead to assembly errors entering subsequent production stages and causing material losses. Summary of the Invention

[0005] To address this, the present invention provides an automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition. This method constructs a system-level assembly topology through multi-angle image acquisition and AI visual recognition, and compares it item by item with the standard process sequence to overcome the problem in the prior art that assembly errors such as incorrect or reversed pipe connections are difficult to automatically intercept before operation due to the lack of automatic identification capability for the overall connection status of the system.

[0006] To achieve the above objectives, this invention provides an automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition, comprising: Based on the visual recognition model, the multi-angle image data of the acquired disposable liquid preparation system after assembly are identified to determine the component type and spatial position of each component in the disposable liquid preparation system, so as to obtain the component identification result; Based on the component identification results, the interface connection relationship between each component and the connection status of the delivery pipeline are identified to determine the actual assembly relationship corresponding to each delivery pipeline. Based on the actual assembly relationship, determine the connection direction and process connection sequence of the conveying pipelines between each component, so as to determine the connection sequence information corresponding to the assembly state of each conveying pipeline; The connection sequence information is matched with the preset standard assembly information. Based on the matching result, the assembly anomaly location and anomaly type corresponding to the current disposable liquid preparation system are determined. The preset standard assembly information includes the installation relationship of the components and the standard connection sequence that meets the liquid preparation process requirements. Based on the location and type of assembly anomalies, error prevention control instructions are generated, and the disposable liquid preparation system is controlled to perform anomaly prompts or operation blocking operations.

[0007] Furthermore, the process of acquiring multi-angle image data after the disposable solution preparation system is assembled includes: Using several industrial cameras positioned above, to the side, and at the interface of the disposable liquid preparation system, images are captured of the overall liquid preparation bag area, the pipeline layout area, and the connection areas of the components. Specifically, the upper industrial camera is used to capture images of the liquid preparation bag placement and the direction of the delivery pipeline, the side industrial camera is used to capture images of the filter components and the side pipeline connections, and the interface area industrial camera is used to capture images of the interface connection status.

[0008] Furthermore, the process of obtaining component identification results includes: Multi-angle image data are input into the visual recognition model to detect and segment each component in each image data, and output the category label and pixel position information of each component in each image data. Each component includes liquid preparation bag, delivery pipeline, sterile connector and filter assembly. Based on the component types of each component, the liquid preparation bag, sterile connector, and filter assembly are regarded as rigid target components, and the delivery pipeline is regarded as a flexible target component. Cross-view correlation matching is performed on different types of target components to determine the same component in the image data of each view. Based on the centroid pixel coordinates of the same component in the image data from different perspectives, the three-dimensional position coordinates of the corresponding component in the space where the disposable liquid preparation system is located are determined. The three-dimensional position coordinates of each component are matched with the preset installation areas to establish the preset component category constraints corresponding to the installation areas of each component. The installation areas include the liquid bag suspension area, the arrangement area of ​​each delivery pipeline, the filter assembly installation area, and the docking area of ​​each sterile connector. When the recognition results of the same component in different image data are inconsistent, the final component type of the corresponding component is determined based on the category label of the corresponding component in each image data and the preset component category constraint corresponding to the installation area of ​​the corresponding component, so as to determine the component recognition result.

[0009] Furthermore, the process of determining the actual assembly relationships includes: Based on the component identification results of each component, the components classified as conveying pipelines are selected from all components, and the image regions and three-dimensional spatial coordinates corresponding to each conveying pipeline are extracted. Identify the locations of each sterile connector, valve, and filter component along the extension direction of the three-dimensional spatial coordinates of each delivery pipeline. Based on the positions of each component along each delivery pipeline, the connection status of the interfaces at both ends of each delivery pipeline is determined to obtain the actual assembly relationship of each delivery pipeline.

[0010] Furthermore, the process of determining the components connected at both ends of the delivery pipeline includes: Based on the three-dimensional spatial coordinates of each delivery pipeline and the three-dimensional spatial coordinates of other components that have been connected, the connection area between the corresponding delivery pipeline and the corresponding component is determined. Edge detection is performed on the edges of the mating surfaces, the inner edges of the mating surfaces, and the snap edges in the connection area based on multi-angle image data to determine whether the corresponding conveying pipeline and the corresponding component have been connected. The connected conveying pipelines are recorded to obtain the interface connection status at both ends of each conveying pipeline. The interface connection status includes: closed locking status, connection abnormal status, and locking misalignment status.

[0011] Furthermore, the process of determining the connection sequence information corresponding to the assembly status of each delivery pipeline includes: Based on the actual assembly relationship of each delivery pipeline, starting from the liquid distribution bag, the liquid flow direction of each delivery pipeline is determined by traversing the connection relationship between the delivery pipeline and each component level by level along the liquid delivery direction. During the traversal, when passing through a filter component, the direction of liquid inflow and outflow from the corresponding filter component is determined based on the delivery pipelines connected to the inlet and outlet ends of the corresponding filter component, respectively. Record the order in which each component is passed by the liquid according to the traversal order to generate the connection sequence information corresponding to the current assembly state.

[0012] Furthermore, the process of matching the connection sequence information with the preset standard assembly information includes: Based on the current liquid preparation process stage, the standard connection sequence corresponding to the current liquid preparation process stage is retrieved from the standard assembly information database. The standard connection sequence is an ordered sequence of each component according to the liquid delivery direction. The connection sequence information corresponding to the current assembly state is compared item by item with the standard connection sequence to determine the matching result. The matching result includes the presence of missing components, inconsistencies in components, and reversed liquid delivery directions.

[0013] Furthermore, the process of determining the location and type of assembly anomaly corresponding to the current disposable solution preparation system includes: In response to the matching result indicating that there is an inconsistency in the components, the three-dimensional spatial coordinates of the corresponding components in the connection sequence information corresponding to the current assembly state are taken as the corresponding assembly anomaly location, and the anomaly type is determined to be a pipe misconnection. In response to the matching result indicating that a component is missing, the three-dimensional spatial coordinates of the corresponding component in the standard connection sequence are used as the corresponding assembly anomaly location, and the anomaly type is determined to be a missing pipe connection. In response to the matching result indicating that there is an opposite liquid delivery direction, the three-dimensional spatial coordinates of the corresponding component in the connection sequence information corresponding to the current assembly state are taken as the corresponding assembly anomaly location, and the anomaly type is determined to be pipeline reverse connection; In response to any interface connection status being either a connection error or a locking misalignment, the three-dimensional spatial coordinates of the corresponding buckle are used as the corresponding assembly error location, and the error type is determined to be a buckle connection error.

[0014] Furthermore, the process of determining the location and type of assembly anomaly corresponding to the current disposable solution preparation system also includes: In the process of determining the component recognition result, when the category with the most numbers in the category labels of the corresponding component in each image data does not meet the preset category constraints of the component, it is determined that the type of the corresponding component has an recognition abnormality. In response to an identification anomaly in the type of a component, the three-dimensional spatial coordinates of the corresponding component are recorded as the location of the assembly anomaly, and the anomaly type is determined to be a pipe misconnection.

[0015] Furthermore, the process of executing error messages or performing blocking operations includes: In response to an abnormality, a start-prohibition command is sent to the delivery pump, and the three-dimensional coordinates of the abnormal assembly location and the specific abnormality type are output to the human-machine interface as an abnormality prompt. Corresponding assembly image data and verification records are also generated.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention first spatially locates the components of the disposable liquid preparation system, such as the liquid preparation bag, delivery pipeline, filter assembly, and sterile connector, based on multi-angle visual acquisition results. Then, it restores the actual connection relationship after assembly based on the spatial extension relationship of the delivery pipeline and the interface connection status. On this basis, the actual connection relationship is converted into connection sequence information reflecting the liquid delivery path and compared item by item with the standard connection sequence under the corresponding liquid preparation process stage. This not only identifies local assembly anomalies such as missing components and incorrect connection objects, but also further identifies assembly anomalies based on the overall flow path relationship, such as incorrect liquid delivery direction and reversed pipeline connection. When an anomaly is detected, an error prevention control command is generated according to the spatial coordinates corresponding to the location of the anomaly, so as to intercept the incorrect assembly state in advance. Attached Figure Description

[0017] Figure 1This is a flowchart of the automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition, as described in this invention. Figure 2 This is a flowchart illustrating how the present invention associates flexible target components from different perspectives into the same component; Figure 3 This is a determination diagram for the connection between other components and corresponding delivery pipelines in this invention; Figure 4 This is a determination diagram for judging the connection status of the latches at both ends of the delivery pipeline according to the present invention. Detailed Implementation

[0018] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0019] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0020] Please see Figure 1 The flowchart shown is a process for an automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition, according to the present invention, including: Step S1: Based on the visual recognition model, identify the multi-angle image data of the acquired disposable liquid preparation system after assembly, determine the component type and spatial position of each component in the disposable liquid preparation system, and obtain the component identification result.

[0021] Step S2: Based on the component identification results, identify the interface connection relationship between each component and the connection status of the delivery pipeline to determine the actual assembly relationship corresponding to each delivery pipeline.

[0022] Step S3: Determine the connection direction and process connection sequence of the conveying pipelines between each component based on the actual assembly relationship, so as to determine the connection sequence information corresponding to the assembly state of each conveying pipeline.

[0023] Step S4: Match the connection sequence information with the preset standard assembly information, and determine the assembly anomaly location and anomaly type corresponding to the current disposable liquid preparation system based on the matching result. The preset standard assembly information includes the installation relationship of the components and the standard connection sequence that meets the liquid preparation process requirements.

[0024] Step S5: Generate error prevention control instructions based on the location and type of assembly anomalies, and control the disposable liquid preparation system to perform anomaly prompts or operation blocking operations. Error prevention control instructions include prohibiting the start of the delivery pump, outputting anomaly prompts to the human-machine interface, and generating corresponding assembly image data and verification records.

[0025] In this embodiment, the process of acquiring multi-angle image data after the disposable solution preparation system is assembled includes: An industrial camera, vertically positioned directly above the dispensing bag in a disposable dispensing system, acquires image data of the bag's placement, the delivery pipeline extending outward from the bag, and its direction.

[0026] Using several industrial cameras tilted at an angle above the disposable liquid preparation system, image data of the filter components, sterile connectors, and all connected delivery pipelines and their directions are acquired from top to bottom at an angle.

[0027] Several industrial cameras, horizontally positioned around the disposable liquid preparation system, acquire image data of the filter components, sterile connectors, and all connected delivery pipelines and their routing.

[0028] Each industrial camera is equipped with an LED ring light source to enhance the clarity of capturing transparent pipes. Each industrial camera simultaneously acquires images and includes its own three-dimensional spatial coordinate data to provide a basis for visual recognition models.

[0029] Specifically, this invention first spatially locates the components of a disposable liquid preparation system, such as the preparation bag, delivery pipeline, filter assembly, and sterile connector, based on multi-angle visual acquisition results. Then, it reconstructs the actual connection relationships after assembly based on the spatial extension relationship of the delivery pipeline and the interface connection status. On this basis, the actual connection relationships are converted into connection sequence information reflecting the liquid delivery path, and compared item by item with the standard connection sequence under the corresponding liquid preparation process stage. This not only identifies local assembly anomalies such as missing components and incorrect connections, but also further identifies assembly anomalies based on the overall flow path relationship, such as incorrect liquid delivery direction and reversed pipeline connections. When an anomaly is detected, an error prevention control command is generated based on the spatial coordinates corresponding to the location of the anomaly, enabling early interception of erroneous assembly states.

[0030] In this embodiment, the process of obtaining the component identification result in step S1 includes: Step S11: Input the multi-angle image data into the visual recognition model respectively, detect and segment each component in each image data, and output the category label and pixel position information of each component in each image data. Each component includes a liquid bag, delivery pipeline, sterile connector and filter assembly. The pixel position information includes instance segmentation mask and centroid pixel coordinates. The instance segmentation mask is a set of multiple pixel coordinates to determine the total pixel area occupied by the corresponding component in the image. The centroid pixel coordinates are determined based on the instance segmentation mask to serve as the representative position of the corresponding component in the corresponding viewpoint image.

[0031] In this embodiment, the visual recognition model adopts an instance segmentation model based on a deep convolutional neural network, which includes a feature extraction backbone network, a region candidate network, a ROIAlign layer, and a classification and segmentation branch. The feature extraction backbone network adopts a combination structure of ResNet-101 and feature pyramid network to receive input image data and output multi-scale feature maps. The region candidate network is used to generate candidate regions on the multi-scale feature maps, and the image corresponding to the candidate region is an image region that may contain components. The ROIAlign layer is used to extract fixed-size feature vectors in each candidate region. The classification and segmentation branch is used to classify and segment each candidate region, and output the category label, category confidence, and pixel-level instance segmentation mask corresponding to each component in each image data. The components include a liquid preparation bag, delivery pipeline, sterile connector, and filter assembly. By inputting multi-angle image data into the visual recognition model, the model outputs the category label corresponding to each component in each image data and the pixel position information in the image.

[0032] In this embodiment, the training process of the visual recognition model includes: pre-collecting multi-angle images of the disposable liquid preparation system under various assembly states in different environments. The assembly states include normal connection, loose connection, misconnection, missing connection, reverse connection, and other abnormal connection states. Each component in each image is outlined at the pixel level and assigned a corresponding category label. The labeled image data is divided into a training set and a validation set. The model is trained using a stochastic gradient descent optimizer with cross-entropy loss and mask loss as the joint loss function. Training stops when the loss value on the validation set tends to stabilize, resulting in a trained visual recognition model used for inference and recognition of input multi-angle image data.

[0033] Step S12: Based on the component type of each component, the liquid preparation bag, sterile connector, and filter assembly are taken as rigid target components, and the delivery pipeline is taken as a flexible target component. For rigid target components, based on the centroid pixel coordinates of each rigid target component in the image data of each viewpoint, for any two industrial cameras, the centroid pixel coordinates of a certain component in the image data of the first viewpoint are used as the reference. Combining the three-dimensional spatial coordinate data, intrinsic parameter matrix, extrinsic parameter matrix, and epipolar geometric relationship of the two industrial cameras, the epipolar equation corresponding to the centroid pixel coordinates in the image data of the second viewpoint is calculated to obtain the corresponding epipolar line. Components whose distance from the centroid pixel coordinates in the second viewpoint to the corresponding epipolar line is less than the preset epipolar line distance are taken as candidate matching objects of the corresponding first viewpoint components, and the closest candidate is selected as the matching target. Rigid target components from different viewpoints that are successfully matched are associated as the same component. The intrinsic parameter matrix includes the focal length and principal point coordinates of the corresponding industrial camera, and the extrinsic parameter matrix includes the rotation matrix and translation vector of the corresponding industrial camera. The intrinsic parameter matrix of each industrial camera and the rotation matrix and translation vector relative to the system base coordinate system are obtained in advance through Zhang Zhengyou calibration method.

[0034] Please see Figure 2 As shown, this is a flowchart of the present invention that associates flexible target components from different perspectives as the same component. Step S13: For the flexible target component, within the instance segmentation mask of the image data from each perspective, the Zhang-Suen thinning algorithm is used to extract the central skeleton line with a single pixel width. Multiple sampling points are extracted along the skeleton line from one end to the other at equal intervals with a fixed pixel step size. The pixel coordinates of each sampling point are used as a local matching reference. For a certain sampling point on the skeleton line in the first perspective image, the corresponding epipolar equation is calculated in the second perspective image based on epipolar geometry. All sampling points are traversed along the central skeleton line of the second perspective image, and the vertical distance from each sampling point to the corresponding epipolar line is calculated. Sampling points with a distance less than the preset epipolar distance are determined as candidate matching points with the same name. When the second perspective image... When multiple skeleton lines exist near the corresponding epipolar line and all sampling points satisfy the condition that the distance between them is less than the preset epipolar line distance, i.e., when there are intersecting skeleton lines, the extension direction of the skeleton line adjacent to the sampling point in the second view and confirmed as the same delivery pipeline in the first view is used as the reference direction. The sampling point with the smallest deviation from the reference direction among the candidate matching points is determined as the same matching point of the first view sampling point. The skeleton line to which the corresponding candidate matching point belongs is determined as the same skeleton line of the pipeline where the first view sampling point is located in the second view. The direction of the line connecting the sampling points on its front and back sides is used as the reference direction. Matching is performed point by point along the skeleton line so that the correct corresponding point relationship is established for each local arc segment of the flexible pipeline in each view, so as to associate the flexible target components in different views as the same component.

[0035] In this embodiment, the preset epipolar distance is used as a criterion for determining whether two components in the image data acquired by different industrial cameras are the same component. It is determined based on the statistical data of the epipolar distance from the centroid pixel coordinates of each image data to the corresponding epipolar line when each industrial camera associates preset calibration objects placed at different positions with different perspectives. This is used to determine the benchmark error of the image data acquired by each industrial camera and the attached three-dimensional spatial coordinate data and epipolar geometric relationship.

[0036] Specifically, by using the central skeleton line of the flexible delivery pipeline as a continuous matching benchmark for cross-view association, and further filtering candidate matching points in areas with intersections by combining the extension direction of local arc segments, the problem of bending, shifting, and difficulty in maintaining correspondence when the same pipeline overlaps with other pipelines due to the flexibility of the disposable delivery pipeline itself can be avoided. This enables the same delivery pipeline in images from different viewpoints to establish a continuous and accurate association.

[0037] Step S14: Based on the centroid pixel coordinates of the same component in the image data from each viewpoint, the centroid pixel coordinates of each component in each image data are back-projected to three-dimensional space using the direct linear transformation method, and the three-dimensional position coordinates of each component in the space where the disposable liquid preparation system is located are obtained.

[0038] Step S15: Match the three-dimensional position coordinates of each component with the spatial range of multiple pre-divided installation areas to determine the installation area to which the corresponding component belongs. Based on the preset allowed component types of the corresponding installation area, establish preset component category constraints for each component corresponding to its installation area. The installation area includes the liquid bag suspension area, the arrangement area of ​​each delivery pipeline, the filter assembly installation area, and the docking area of ​​each sterile connector.

[0039] In this embodiment, the installation area is a plurality of corresponding three-dimensional spatial regions pre-defined in a three-dimensional spatial coordinate system according to the structural layout of the disposable liquid preparation system. Each installation area has its own coordinate range boundary values ​​in the X-axis, Y-axis, and Z-axis directions. For the same component, the three-dimensional position coordinates of the corresponding component are compared with the coordinate range boundary values ​​of each installation area. When the X-axis coordinate value, Y-axis coordinate value, and Z-axis coordinate value of the corresponding component are respectively within the X-axis, Y-axis, and Z-axis coordinate ranges of a certain installation area, it is determined that the corresponding component belongs to the corresponding installation area. The above matching operation is repeated until the installation area to which all components belong is determined.

[0040] In this embodiment, the permitted component types are a set of permitted component types for each installation area, which are determined based on the structural layout and liquid preparation process requirements of the disposable liquid preparation system. Specifically, the permitted component types in the liquid preparation bag suspension area include liquid preparation bags and delivery pipelines; the permitted component types in the filter assembly installation area include filter assemblies and delivery pipelines; the permitted component types in the aseptic connector docking area include aseptic connectors and delivery pipelines; and the permitted component types in the delivery pipeline arrangement area include delivery pipelines.

[0041] Step S16: When the category labels corresponding to the same component are inconsistent in different image data, the category with the most categories in each image data is determined as the final category label. When the final category label of the component satisfies the preset component category constraint corresponding to the installation area, the final component type of the corresponding component is determined, so as to determine the final component type of each component as the component recognition result.

[0042] Specifically, this invention distinguishes the constituent components into rigid target components and flexible target components. For rigid target components, the overall centroid is used as the matching benchmark. For flexible target components, the central skeleton line is extracted and cross-view matching is performed point by point. This solves the problem that conventional feature point matching methods are prone to failure in the prior art due to the uncertain shape of flexible pipelines. This enables visual recognition to stably establish cross-view correspondence in mixed assembly scenarios that include flexible pipelines and rigid components.

[0043] Please see Figure 3 As shown, this is a determination diagram for the connection between other components and corresponding conveying pipelines in this invention. In this embodiment, the process of determining the actual assembly relationship in step S2 includes: Step S21: Based on the component identification results and corresponding three-dimensional position coordinates of each component, filter out the components that are classified as conveying pipelines from all components, and extract the image area and three-dimensional spatial coordinate point set corresponding to each conveying pipeline.

[0044] Step S22: Based on the results of skeleton extraction using the Zhang-Suen thinning algorithm on the image regions corresponding to each delivery pipeline, determine the extension direction of the delivery pipeline and the arrangement order of each coordinate point along the extension direction, and determine the endpoint coordinates of the two ends of the corresponding delivery pipeline.

[0045] Step S23: Along the extension direction of the corresponding delivery pipeline, with each endpoint coordinate point as the center and a preset spatial distance as the radius, a search range is formed to check whether the three-dimensional position coordinates of other components, namely sterile connectors, valves or filter components, are within the preset spatial distance of the coordinate points of the delivery pipeline endpoints. If they fall within the range, it is determined that the component at the corresponding coordinate point is connected to the corresponding delivery pipeline, so as to record the positions of all components passed by each delivery pipeline and the order of each component in the extension direction.

[0046] In this embodiment, the preset spatial distance is used as the criterion for determining whether the delivery pipeline passes through the position of a certain component. It is determined based on the statistical distribution of the three-dimensional spatial distance between the pipeline and each component after the standard assembly of the disposable liquid preparation system is completed in historical data, so as to determine the spatial positioning error tolerance when identifying the position of the component based on three-dimensional spatial coordinates.

[0047] Step S24: Based on the positions of each component along each conveying pipeline, determine the interface connection status at both ends of each conveying pipeline to obtain the actual assembly relationship of each conveying pipeline.

[0048] Specifically, this invention can directly output the position of the component connected to each pipeline endpoint by automatically separating the independent direction of each delivery pipeline from all components, providing complete pipeline topology information for determining assembly relationships. Furthermore, by searching the position of each component along the pipeline extension direction and recording the order, the pipeline connection relationship can be accurately determined automatically.

[0049] In this embodiment, the process of determining the components connected to both ends of the delivery pipeline in step S24 includes: Step S241: Based on the coordinates of the endpoints at both ends of the conveying pipeline, search for the component closest to each endpoint to determine the connection area between each end of the conveying pipeline and the corresponding connected component. Select all the image data covering the mating surface of the corresponding connection area from the multi-angle image data, and extract a local sub-image of the corresponding connection area from the corresponding image data.

[0050] Step S242: Use the Canny edge detection algorithm to perform edge detection on each local sub-image and extract all edge contours in the local sub-image, which includes at least the insertion mating surface edge and the snap-fit ​​edge. The insertion mating surface edge is the edge of the end of the conveying pipeline in the local sub-image, located on the side of the local sub-image closer to the end face of the component it is connected to; the snap-fit ​​edge is the edge of the flip-locking structure at the interface of the connecting component, located on the outer periphery of the interface of the connecting component.

[0051] Step S243: Using the position of the edge of the mating surface in the local sub-image as the reference position, the position obtained by offsetting the three-dimensional position coordinates of the connected component inward along the connection direction by a preset length is taken as the theoretical position of the inner edge of the mating surface, and the inner edge of the mating surface is determined accordingly. The pixel distance between the edge of the mating surface and the inner edge of the mating surface along the connection direction is the pixel distance corresponding to the insertion depth.

[0052] In this embodiment, the preset length is the standard depth dimension between the end face of the interface of the connecting component and the bottom wall inside the interface, which is determined based on the physical structural dimensions of the connected component.

[0053] Step S244: When there is an edge profile between the edge of the mating surface and the inner edge of the mating surface that matches the distance between adjacent edges and the width of the snap fastener and is approximately perpendicular to the interface axis, the corresponding edge profile is determined as the snap fastener edge; otherwise, it is determined that the snap fastener is in a locked misaligned state, indicating that the connection between the conveying pipeline and the corresponding connecting component is not completed.

[0054] In this embodiment, the buckle width of each connecting component is determined based on the dimensions of the physical structure of the connecting component, the position of the corresponding industrial camera, and the viewing angle of the actual shooting position.

[0055] Please see Figure 4 As shown, this is a determination diagram for judging the connection status of the buckles at both ends of the conveying pipeline according to the present invention. Step S245: Determine whether the buckle is in the closed and locked position based on the geometric shape characteristics of the buckle edge in each image data: When the buckle edge presents a continuous strip-shaped outline that is approximately perpendicular to the interface axis, if the aspect ratio of the smallest bounding rectangle of the continuous strip-shaped outline is greater than the preset aspect ratio threshold, and the proportion of the continuous edge length to the standard perimeter of the buckle is greater than the preset length proportion threshold, the buckle is determined to be in the closed and locked state. At this time, the buckle has locked the pipeline in the connecting component. If a buckle in any image data is determined to be not in the closed and locked position, the corresponding buckle is determined to be in an abnormal connection state.

[0056] In this embodiment, the preset aspect ratio threshold and the preset length ratio threshold are determined based on the physical structure dimensions of the buckle in the locked state and the viewing angle between each industrial camera and its corresponding position.

[0057] Step S246: Record the components and connection status that have been connected at both ends of each delivery pipeline to obtain the interface connection status at both ends of each delivery pipeline.

[0058] Specifically, this invention simultaneously detects the mating surface and the locking status of the buckle from the same local sub-image after a single image acquisition. It also reduces the impact of pipe end obstruction on the identification of the insertion depth by using a preset length of the three-dimensional position coordinate offset of the connected components as the basis for determining the insertion depth. This allows for connection status verification even when the obstructed part of the connector may not be accurately identified. Finally, it identifies whether the buckle has completed the closing and locking by using image data from each perspective.

[0059] In this embodiment, the process of determining the connection sequence information corresponding to the assembly state of each delivery pipeline in step S3 includes: Step S31: Based on the actual assembly relationship of each delivery pipeline, starting from the liquid distribution bag, traverse the pipeline level by level according to the connection relationship between the delivery pipeline and each component, determine the liquid flow direction of each delivery pipeline along the liquid delivery direction, so as to construct the connection topology diagram of the disposable liquid distribution system.

[0060] Step S32: Starting from the node corresponding to the liquid preparation bag, use the depth-first traversal algorithm to traverse the nodes in the connection topology graph level by level along the delivery pipeline.

[0061] Step S33: For the currently traversed node, obtain all adjacent nodes connected to the corresponding node, and determine the liquid flow direction according to the type of component to which the corresponding node belongs. Specifically, if the current node is a liquid distribution bag, determine that the liquid flows from the liquid distribution bag to the delivery pipeline connected to it; if the current node is a delivery pipeline, determine the liquid delivery direction according to the components connected to both ends of the delivery pipeline; if the current node is a filter assembly, determine the inlet and outlet ends based on the interface identification direction of the filter assembly, and determine the liquid inflow side and liquid outflow side in combination with the positional relationship of the delivery pipelines connected to both sides of the filter assembly, so that the liquid flows into the filter assembly from the delivery pipeline corresponding to the inlet end and flows out from the delivery pipeline corresponding to the outlet end; if there are branch nodes in the pipeline, record the flow path independently along each branch direction until the terminal node is traversed, and record the order in which each node is visited and the liquid flow direction of each delivery pipeline during the traversal process.

[0062] In this embodiment, the inlet and outlet ends of the filter assembly are fixed interface direction information predetermined according to the structural design of the filter assembly. This information can be determined through flow direction markings, interface numbers, or assembly structural features on the filter assembly housing. Since the interfaces on both sides of the filter assembly have a fixed flow direction relationship, determining the inlet and outlet ends of the filter assembly avoids the problem of not being able to determine the filtration direction based solely on the pipeline spatial connection relationship, thereby ensuring that subsequent connection sequence information accurately reflects the actual liquid delivery path.

[0063] Step S34: Record the order in which each component is passed by the liquid according to the traversal order, so as to generate the connection sequence information corresponding to the current assembly state.

[0064] Specifically, this invention transforms the actual assembly relationship of each delivery pipeline into connection sequence information that reflects the liquid flow sequence, and records the independent flow direction of each branch path when there are branch pipelines, so that the generated connection sequence information covers all liquid flow paths starting from the liquid bag in the disposable liquid preparation system, providing a data basis for subsequent identification of errors such as reversed flow direction or connection sequence that does not meet the process path requirements.

[0065] In this embodiment, the process of matching the connection sequence information with the preset standard assembly information in step S4 includes: Step S41: Based on the current liquid preparation process stage, retrieve the standard connection sequence corresponding to the current liquid preparation process stage from the standard assembly information database. The standard connection sequence is an ordered sequence of each component according to the liquid delivery direction.

[0066] In this embodiment, the standard assembly information database is a structured database pre-stored in an industrial control computer or edge computing system. Its construction process includes: for each liquid preparation process stage, determining the standard flow path between the components of the liquid in the disposable liquid preparation system according to the corresponding liquid preparation process requirements. The standard flow path is an ordered sequence of components that the liquid passes through sequentially from the preparation bag to the target receiving container. The component type, component number, connection relationship between components, and liquid flow direction of each delivery pipeline along the standard flow path are organized to generate the standard connection sequence corresponding to the liquid preparation process stage. Finally, all standard connection sequences corresponding to all liquid preparation process stages are classified and stored according to the process stage identifier, and an association index is established between each standard connection sequence and the corresponding liquid preparation process stage. This allows for the retrieval of the corresponding standard connection sequence through the association index during subsequent verification based on the current liquid preparation process stage. The standard flow path is determined based on the process flow document and assembly design document corresponding to the disposable liquid preparation system. Specifically, it is determined according to the process connection requirements between the liquid preparation bag, filter assembly, sterile connector and target receiving container, the order of components through which the liquid is allowed to pass and the flow direction relationship of each delivery pipeline are determined. When there are disposable liquid preparation systems of different specifications in the same liquid preparation process stage, an independent standard connection sequence is established according to the corresponding system model to ensure that the standard connection sequence called in the subsequent verification process matches the current assembly object.

[0067] Step S42: Based on the connection sequence information corresponding to the current assembly state, compare it item by item with the standard connection sequence. If the component corresponding to a certain position in the standard connection sequence does not exist in the same position in the connection sequence information corresponding to the current assembly state, determine that the matching result of the corresponding position is that a component is missing. If the type of the component corresponding to a certain position in the connection sequence information corresponding to the current assembly state is inconsistent with the type of the component at the same position in the standard connection sequence, determine that the matching result of the corresponding position is that a component is inconsistent. If the liquid flow direction of a certain conveying pipeline in the connection sequence information corresponding to the current assembly state is opposite to the liquid flow direction of the corresponding conveying pipeline in the standard connection sequence, determine that the matching result of the corresponding position is that the liquid conveying direction is opposite.

[0068] In this embodiment, the process of determining the assembly anomaly location and anomaly type corresponding to the current disposable solution preparation system in step S4 includes: Step S43: In response to the matching result indicating that there is an inconsistency in the components, the three-dimensional spatial coordinates of the corresponding components in the connection sequence information corresponding to the current assembly state are taken as the corresponding assembly anomaly location, and the anomaly type is determined to be a pipe misconnection.

[0069] Step S44: In response to the matching result indicating that a component is missing, the three-dimensional spatial coordinates of the corresponding component in the standard connection sequence are taken as the corresponding assembly anomaly location, and the anomaly type is determined to be a missing pipe connection.

[0070] Step S45: In response to the matching result indicating that there is an opposite liquid delivery direction, the three-dimensional spatial coordinates of the corresponding component in the connection sequence information corresponding to the current assembly state are taken as the corresponding assembly abnormality position, and the abnormality type is determined to be pipeline reverse connection.

[0071] Step S46: In response to any interface connection state being either a connection error state or a locking misalignment state, the three-dimensional spatial coordinates of the corresponding buckle are taken as the corresponding assembly error position, and the error type is determined to be a buckle connection error.

[0072] In this embodiment, the process of determining the assembly anomaly location and anomaly type corresponding to the current disposable solution preparation system in step S4 further includes: Step S47: In the process of determining the component recognition result, when the category with the most numbers of category labels for the corresponding component in each image data does not meet the preset category constraints for the component, it is determined that the type of the corresponding component has an recognition anomaly.

[0073] Step S48: In response to the identification anomaly of the component type, record the three-dimensional spatial coordinates of the corresponding component as the corresponding assembly anomaly location, and determine the anomaly type as pipeline misconnection.

[0074] Specifically, this invention restores the actual flow path structure of a disposable liquid preparation system after assembly by progressively associating the spatial positions of components, pipeline connections, and interface states obtained from multi-angle visual recognition. The corresponding actual flow path structure is then converted into connection sequence information and matched with the standard process flow path. This extends the recognition from local connections to overall delivery path verification. It can not only identify local assembly anomalies such as incorrect or missing connections, but also identify pipeline reversal anomalies caused by incorrect connection directions. Based on the spatial position of the anomaly, it generates error prevention control commands to automatically intercept erroneous assembly states before liquid preparation begins.

[0075] In this embodiment, the process of executing the abnormal prompt or running the blocking operation in step S5 includes: Step S51: In response to the occurrence of an abnormality type, a start prohibition command is sent to the delivery pump, and the three-dimensional coordinates of the assembly abnormality location and the specific abnormality type are output to the human-machine interface as an abnormality prompt, and the corresponding assembly image data and verification record are generated.

[0076] Specifically, when any type of abnormality is detected, the present invention automatically sends a prohibition command to the delivery pump to block the operation of the liquid preparation system, thereby reducing the probability of economic losses. At the same time, it outputs the abnormality type and its corresponding assembly abnormality location coordinates to the human-machine interface, and simultaneously generates an archive file containing assembly image data and verification records, providing data support for tracing the cause of the abnormality and reviewing the operation after it occurs.

[0077] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An automatic verification and error prevention method for a disposable liquid dispensing system based on AI visual recognition, characterized in that, include: Based on the visual recognition model, the multi-angle image data of the acquired disposable liquid preparation system after assembly are identified to determine the component type and spatial position of each component in the disposable liquid preparation system, so as to obtain the component identification result; Based on the component identification results, the interface connection relationship between each component and the connection status of the delivery pipeline are identified to determine the actual assembly relationship corresponding to each delivery pipeline. Based on the actual assembly relationship, determine the connection direction and process connection sequence of the conveying pipelines between each component, so as to determine the connection sequence information corresponding to the assembly state of each conveying pipeline; The connection sequence information is matched with the preset standard assembly information. Based on the matching result, the assembly anomaly location and anomaly type corresponding to the current disposable liquid preparation system are determined. The preset standard assembly information includes the installation relationship of the components and the standard connection sequence that meets the liquid preparation process requirements. Based on the location and type of assembly anomalies, error prevention control instructions are generated, and the disposable liquid preparation system is controlled to perform anomaly prompts or operation blocking operations.

2. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition as described in claim 1, characterized in that, The process of acquiring multi-angle image data after the assembly of a disposable solution preparation system includes: Using several industrial cameras positioned above, to the side, and at the interface of the disposable liquid preparation system, images are captured of the overall liquid preparation bag area, the pipeline layout area, and the connection areas of the components. Specifically, the upper industrial camera is used to capture images of the liquid preparation bag placement and the direction of the delivery pipeline, the side industrial camera is used to capture images of the filter components and the side pipeline connections, and the interface area industrial camera is used to capture images of the interface connection status.

3. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition according to claim 2, characterized in that, The process of obtaining component identification results includes: Multi-angle image data are input into the visual recognition model to detect and segment each component in each image data, and output the category label and pixel position information of each component in each image data. Each component includes liquid preparation bag, delivery pipeline, sterile connector and filter assembly. Based on the component types of each component, the liquid preparation bag, sterile connector, and filter assembly are regarded as rigid target components, and the delivery pipeline is regarded as a flexible target component. Cross-view correlation matching is performed on different types of target components to determine the same component in the image data of each view. Based on the centroid pixel coordinates of the same component in the image data from different perspectives, the three-dimensional position coordinates of the corresponding component in the space where the disposable liquid preparation system is located are determined. The three-dimensional position coordinates of each component are matched with the preset installation areas to establish the preset component category constraints corresponding to the installation areas of each component. The installation areas include the liquid bag suspension area, the arrangement area of ​​each delivery pipeline, the filter assembly installation area, and the docking area of ​​each sterile connector. When the recognition results of the same component in different image data are inconsistent, the final component type of the corresponding component is determined based on the category label of the corresponding component in each image data and the preset component category constraint corresponding to the installation area of ​​the corresponding component, so as to determine the component recognition result.

4. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition according to claim 3, characterized in that, The process of determining the actual assembly relationships includes: Based on the component identification results of each component, the components classified as conveying pipelines are selected from all components, and the image regions and three-dimensional spatial coordinates corresponding to each conveying pipeline are extracted. Identify the locations of each sterile connector, valve, and filter component along the extension direction of the three-dimensional spatial coordinates of each delivery pipeline. Based on the positions of each component along each delivery pipeline, the connection status of the interfaces at both ends of each delivery pipeline is determined to obtain the actual assembly relationship of each delivery pipeline.

5. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition according to claim 4, characterized in that, The process of determining the components connected at both ends of the delivery pipeline includes: Based on the three-dimensional spatial coordinates of each delivery pipeline and the three-dimensional spatial coordinates of other components that have been connected, the connection area between the corresponding delivery pipeline and the corresponding component is determined. Edge detection is performed on the edges of the mating surfaces, the inner edges of the mating surfaces, and the snap edges in the connection area based on multi-angle image data to determine whether the corresponding conveying pipeline and the corresponding component have been connected. The connected conveying pipelines are recorded to obtain the interface connection status at both ends of each conveying pipeline. The interface connection status includes: closed locking status, connection abnormal status, and locking misalignment status.

6. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition according to claim 5, characterized in that, The process of determining the connection sequence information corresponding to the assembly status of each delivery pipeline includes: Based on the actual assembly relationship of each delivery pipeline, starting from the liquid distribution bag, the liquid flow direction of each delivery pipeline is determined by traversing the connection relationship between the delivery pipeline and each component level by level along the liquid delivery direction. During the traversal, when passing through a filter component, the direction of liquid inflow and outflow from the corresponding filter component is determined based on the delivery pipelines connected to the inlet and outlet ends of the corresponding filter component, respectively. Record the order in which each component is passed by the liquid according to the traversal order to generate the connection sequence information corresponding to the current assembly state.

7. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition according to claim 6, characterized in that, The process of matching the connection sequence information with the preset standard assembly information includes: Based on the current liquid preparation process stage, the standard connection sequence corresponding to the current liquid preparation process stage is retrieved from the standard assembly information database. The standard connection sequence is an ordered sequence of each component according to the liquid delivery direction. The connection sequence information corresponding to the current assembly state is compared item by item with the standard connection sequence to determine the matching result. The matching result includes the presence of missing components, inconsistencies in components, and reversed liquid delivery directions.

8. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition according to claim 7, characterized in that, The process of determining the location and type of assembly anomaly in the current disposable solution preparation system includes: In response to the matching result indicating that there is an inconsistency in the components, the three-dimensional spatial coordinates of the corresponding components in the connection sequence information corresponding to the current assembly state are taken as the corresponding assembly anomaly location, and the anomaly type is determined to be a pipe misconnection. In response to the matching result indicating that a component is missing, the three-dimensional spatial coordinates of the corresponding component in the standard connection sequence are used as the corresponding assembly anomaly location, and the anomaly type is determined to be a missing pipe connection. In response to the matching result indicating that there is an opposite liquid delivery direction, the three-dimensional spatial coordinates of the corresponding component in the connection sequence information corresponding to the current assembly state are taken as the corresponding assembly anomaly location, and the anomaly type is determined to be pipeline reverse connection; In response to any interface connection status being either a connection error or a locking misalignment, the three-dimensional spatial coordinates of the corresponding buckle are used as the corresponding assembly error location, and the error type is determined to be a buckle connection error.

9. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition according to claim 8, characterized in that, The process of determining the location and type of assembly anomaly in the current disposable solution preparation system also includes: In the process of determining the component recognition result, when the category with the most numbers in the category labels of the corresponding component in each image data does not meet the preset category constraints of the component, it is determined that the type of the corresponding component has an recognition abnormality. In response to an identification anomaly in the type of a component, the three-dimensional spatial coordinates of the corresponding component are recorded as the location of the assembly anomaly, and the anomaly type is determined to be a pipe misconnection.

10. The automatic verification and error prevention method for a disposable liquid preparation system based on AI visual recognition according to claim 8, characterized in that, The process of displaying error messages or performing blocking operations includes: In response to an abnormality, a start-prohibition command is sent to the delivery pump, and the three-dimensional coordinates of the abnormal assembly location and the specific abnormality type are output to the human-machine interface as an abnormality prompt. Corresponding assembly image data and verification records are also generated.

Images