Defect visual tracking method and device

Abstract

The application provides a defect visual tracking method and device, and relates to the technical field of defect detection. The method comprises the following steps: acquiring continuous multiple frames of images of a transparent bottle body to be detected filled with liquid; determining a first frame of image in the continuous multiple frames of images as a reference image; performing defect analysis on each non-reference image based on the reference image to obtain a defect analysis result corresponding to each non-reference image; and performing defect trajectory clustering according to the defect analysis result corresponding to each non-reference image to obtain a defect motion trajectory of the defect of the transparent bottle body to be detected in the continuous multiple frames of images. The defect motion trajectory obtained by the application can reflect the dynamic process of the defect moving with the liquid, which is helpful to determine whether the defect is located inside the liquid or attached to the bottle wall, and can improve the accuracy and reliability of the defect visual tracking.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and more specifically, to a defect visual tracking method and apparatus. Background Technology

[0002] After the transparent bottles in the pharmaceutical industry have completed processes such as filling and capping, a visual inspection system is needed to screen for defects in the liquid inside the bottle in order to ensure the quality of the medicine.

[0003] In existing technologies, defect detection of transparent bottles filled with liquid is mostly based on single-frame image analysis. Suspected defect areas inside the bottle are extracted through threshold segmentation and morphological operations. The presence of defects is determined solely based on single-frame features such as area and shape, which easily leads to false detections and missed detections, making it difficult to meet the high-precision and high-stability detection requirements of industrial sites. Summary of the Invention

[0004] The purpose of this application is to provide a defect visual tracking method and apparatus to solve the above-mentioned problems existing in the prior art and improve the accuracy and reliability of defect visual tracking.

[0005] Firstly, a defect visual tracking method is provided, which may include: While the liquid filling the transparent bottle to be tested moves relative to the transparent bottle to be tested, acquire a series of multiple frames of images of the transparent bottle to be tested. The first frame in a series of consecutive images is determined as the reference image; Based on the reference image, defect analysis is performed on each non-reference image to obtain the defect analysis results corresponding to each non-reference image. Based on the defect analysis results corresponding to each non-reference image, defect trajectory clustering is performed to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

[0006] In an optional implementation, while the liquid filling the transparent bottle to be detected is moving relative to the transparent bottle, a series of multiple frames of images of the transparent bottle to be detected are acquired, including: While the liquid moves relative to the transparent bottle to be detected, acquire multiple consecutive initial images of the transparent bottle to be detected; The first frame of a series of consecutive initial images is used as the initial reference image. Based on the initial reference image, a caliper measurement model is constructed; wherein, the caliper measurement model is used to locate the edges of the initial reference image; Based on the size parameters of the transparent bottle to be tested and the caliper measurement model, the vertex coordinates of the region to be extracted are determined; Based on the vertex coordinates of the region to be extracted, the initial images of multiple consecutive frames are cropped to obtain multiple consecutive frames of images.

[0007] In an optional implementation, based on the reference image, defect analysis is performed on each non-reference image to obtain the defect analysis results corresponding to each non-reference image, including: For any non-reference image, calculate the grayscale difference response, gradient difference response, and local texture difference response of each pixel in the non-reference image and the corresponding pixel in the reference image; The grayscale difference response, gradient difference response, and local texture difference response of each pixel are weighted and fused to obtain the fused differential response of each pixel; Based on the fusion difference response of each pixel, a fusion difference response map corresponding to the non-reference image is generated; Defect analysis is performed on the fused differential response map corresponding to the non-reference image to obtain the defect analysis results corresponding to the non-reference image.

[0008] In an optional implementation, defect analysis is performed on the fused differential response map corresponding to the non-reference image to obtain the defect analysis results corresponding to the non-reference image, including: For any fused differential response map, the region in the fused differential response map where the fused differential response is greater than a preset response threshold is determined as the first defect region; Morphological operations are performed on multiple identified first defect regions to obtain multiple second defect regions; By performing connected component analysis on multiple second defect regions, multiple third defect regions are obtained; The third defect region corresponding to the defect area that is larger than the preset area threshold is determined as the target defect region of the non-reference image. Extract the defect features of each target defect region and construct a defect energy weight model; Based on the defect energy weighting model, calculate the defect center coordinates of each target defect region; The target defect regions of the non-reference image, along with the defect center coordinates and defect features of each target defect region, are used as the defect analysis results corresponding to the non-reference image.

[0009] In an optional implementation, the defect analysis results include each target defect region in the non-reference image, as well as the defect center coordinates and defect features of each target defect region; Based on the defect analysis results corresponding to each non-reference image, defect trajectory clustering is performed to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images, including: Use any non-reference image as the non-reference image of the current frame; Based on the defect center coordinates and defect features of each target defect region in the current frame non-reference image and the defect center coordinates and defect features of each target defect region in the previous frame non-reference image, calculate the comprehensive matching degree between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image. Based on the comprehensive matching degree, the matching relationship between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image is determined; Based on the matching relationship, defect trajectory clustering is performed to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

[0010] In an optional implementation, calculating the comprehensive matching degree between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image includes: The defect center coordinates of each target defect region in the previous non-reference image are predicted by using a pre-built rotational motion prediction model, and the predicted center coordinates of each target defect region in the previous non-reference image are obtained in the current non-reference image. For any target defect region in the current frame non-reference image, the defect distance between the target defect region and each target defect region in the previous frame non-reference image is calculated based on the defect center coordinates of the target defect region and the defect center coordinates of each target defect region in the previous frame non-reference image. Based on the defect features of the target defect region and the defect features of each target defect region in the previous non-reference image, the feature difference between the target defect region and each target defect region in the previous non-reference image is determined. Calculate the difference between the predicted center coordinates and the defect center coordinates of each target defect region to obtain the predicted position deviation between the target defect region and each target defect region in the previous non-reference image. Based on the defect distance, the feature difference, and the predicted position deviation, the comprehensive matching degree between the target defect region and each target defect region in the previous frame non-reference image is calculated.

[0011] In an optional implementation, defect trajectory clustering is performed based on the matching relationship to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images, including: Two target defect regions whose overall matching degree meets the preset matching conditions have been successfully matched. If any target defect region in the current frame non-reference image successfully matches the defect motion trajectory to which any target defect region in the previous frame non-reference image belongs, then the defect center coordinates of the corresponding target defect region in the current frame non-reference image are appended to the successfully matched defect motion trajectory in chronological order, and the trajectory number of the defect motion trajectory in the consecutive frames of images is unified. If any target defect region in the current frame non-reference image fails to match the defect motion trajectory to which any target defect region in the previous frame non-reference image belongs, then a new defect motion trajectory is created for the corresponding target defect region in the current frame non-reference image, and a unique trajectory number is assigned to each defect motion trajectory. By iterating through all non-reference images and performing frame-by-frame recursive matching, the defect motion trajectory of each defect in consecutive multi-frame images is generated.

[0012] In an optional implementation, after obtaining the defect trajectory of the transparent bottle to be detected in multiple consecutive frames of images, the method further includes: Based on the defect motion trajectory and defect features of any defect in the transparent bottle to be detected in multiple consecutive frames of images, calculate the feature fluctuation difference, motion direction, trajectory length and trajectory reliability score of the defect; If the trajectory length, direction of movement, trajectory reliability score, and feature fluctuation difference of the defect all meet the configured preset valid defect conditions, then the defect is determined to be a valid defect. If the trajectory length, direction of movement, trajectory reliability score, or feature fluctuation difference of the defect does not meet the preset valid defect conditions, the defect will be determined as an invalid defect. Based on the defect center coordinates, defect motion trajectory, trajectory number, and trajectory length of each effective defect in multiple consecutive frames of images, the target defect detection results of the transparent bottle to be detected are obtained.

[0013] In an optional implementation, based on the defect motion trajectory and defect features of any defect in the transparent bottle to be detected across multiple consecutive frames, a trajectory confidence score for the defect is calculated, including: The number of consecutive matches is obtained based on the number of defect center coordinates contained in the defect's motion trajectory. Calculate the trajectory continuity score based on the center coordinates of each defect, the characteristic fluctuation difference, and the direction of movement of each defect in the defect's trajectory. Based on the center coordinates of each defect in the defect's motion trajectory and the predicted center coordinates corresponding to each defect center coordinate, calculate the motion consistency score; The trajectory credibility score is calculated based on the number of consecutive matches, the trajectory continuity score, and the motion consistency score.

[0014] Secondly, a defect visual tracking device is provided, which may include: The acquisition unit is used to acquire multiple consecutive frames of images of the transparent bottle to be detected when the liquid filled in the transparent bottle to be detected moves relative to the transparent bottle to be detected. The determining unit is used to determine the first frame of a series of consecutive frames as the reference image; The analysis unit is used to perform defect analysis on each non-reference image based on the reference image, and obtain the defect analysis results corresponding to each non-reference image. The clustering unit is used to cluster defect trajectories based on the defect analysis results corresponding to each non-reference image, so as to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

[0015] This application uses the first frame of a series of images as a baseline to perform defect analysis on subsequent non-baseline images, thus eliminating the need for pre-prepared external standard templates and enabling adaptive detection of defects in transparent bottles filled with liquid. By clustering the defect analysis results of each non-baseline image into trajectories, the motion trajectory of the defect in the series of images can be obtained, thereby distinguishing between real defects and random noise. The obtained defect motion trajectory can reflect the dynamic process of the defect moving with the liquid, which helps to determine whether the defect is located inside the liquid or attached to the bottle wall, improving the reliability of the detection results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 An architecture diagram of a defect visual tracking system provided in this application embodiment; Figure 2 A flowchart illustrating a defect visual tracking method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a defect visual tracking device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art. The words "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are only used to distinguish different components. The words "comprising" or "including," etc., mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, but do not exclude other elements or objects. The words "connected," "coupled," or "connected," etc., are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," "right," etc., are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] The defect visual tracking method provided in this application embodiment can be applied to... Figure 1 In the system architecture shown, such as Figure 1 As shown, the system may include: an image acquisition device and a server; the image acquisition device is used to acquire multiple consecutive frames of images of the transparent bottle to be detected when the liquid filled in the transparent bottle moves relative to the transparent bottle to be detected; the transparent bottle to be detected may be a vial; the server is used to execute the defect visual tracking method provided in the embodiments of this application; the server may be a physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0020] In one embodiment of this application, the number of image acquisition devices can be 1; the image acquisition device can be an industrial camera.

[0021] In another embodiment of this application, the system may further include: a rotating component; a transparent bottle filled with liquid to be tested is fixed on the rotating component; the rotating component is used to control the transparent bottle to be tested to rotate first and then come to rest, so that the liquid in the transparent bottle to be tested moves relative to the transparent bottle to be tested.

[0022] Specifically, the rotating component may include a rotating shaft, a fixed component, and a rotating controller. The rotating controller receives a rotating control command from the server, controlling the rotating shaft to drive the fixed component, which holds the transparent bottle to be tested, to rotate continuously in the same direction for a first preset time, so that the liquid inside the transparent bottle can be fully expanded in 360°. Then, it receives a stationary command from the server, controlling the rotating shaft to stop rotating, so that the transparent bottle to be tested, fixed on the fixed component, stops moving. At this time, the liquid inside the transparent bottle continues to move due to inertia. When the rotating shaft stops rotating for a second preset time, i.e., after the transparent bottle to be tested has been stationary for a second preset time, the server controls the image acquisition device to acquire multiple consecutive frames of images of the transparent bottle filled with liquid, so that the acquired multiple consecutive frames are not focused on the stationary liquid and the stationary transparent bottle to be tested. The first preset time can be 2 seconds; the second preset time can be 1 second. This application first controls the transparent bottle to be tested to be stationary for a second preset time to stabilize the liquid level inside the transparent bottle. At this time, acquiring multiple consecutive frames of images can obtain a full-area image of the liquid, ensuring that there are no blind spots in the detection of defects inside the transparent bottle.

[0023] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0024] Figure 2 This is a flowchart illustrating a defect visual tracking method provided in an embodiment of this application. Figure 2 As shown, the method may include: Step S210: When the liquid filling the transparent bottle to be tested moves relative to the transparent bottle to be tested, acquire multiple consecutive frames of images of the transparent bottle to be tested.

[0025] In practice, while the liquid moves relative to the transparent bottle to be detected, multiple consecutive initial images of the transparent bottle are acquired. Specifically, after the transparent bottle to be detected is fixed to the fixing component, the server sends a rotation control command to the rotation controller of the rotation component, so that the rotation controller controls the rotation axis to drive the fixing component to rotate continuously in any direction for a first preset time. The server then sends a stop control command to the rotation controller of the rotation component, so that the fixing component with the transparent bottle to be detected stops rotating, i.e., the transparent bottle to be detected stops rotating. After the stop rotation time is not less than a second preset time, the image acquisition device is controlled to continuously acquire a sequence of images of a preset number of frames of the transparent bottle to be detected. The preset number of frames can be 40. The first frame of a series of consecutive initial images is used as the initial reference image. Based on the initial reference image, a caliper measurement model is constructed; the caliper measurement model is used to locate the edges of the initial reference image; the caliper measurement model may include the equations of the left and right edges of the transparent bottle to be detected in the initial reference image and the parameters of the caliper measurement model. Based on the size parameters of the transparent bottle to be inspected and the caliper measurement model, the vertex coordinates of the area to be extracted are determined. The size parameters may include the bottle body model and geometric dimensions of the transparent bottle to be inspected. Specifically, based on the bottle body model and geometric dimensions of the transparent bottle to be inspected, a polygonal area that can include the bottle body is determined as the area to be extracted, and the vertex coordinates of the area to be extracted are determined. This area to be extracted is the effective detection area. Based on the vertex coordinates of the region to be extracted, the initial images of multiple consecutive frames are cropped to obtain multiple consecutive frames of images. Specifically, all initial images are cropped using the vertex coordinates of the same region to be extracted, without repeated localization or recalculation, ensuring that the effective detection area of ​​the entire sequence of images is consistent and the detection standard is uniform.

[0026] In another embodiment of this application, after taking the first initial image in a series of consecutive initial images as the initial reference image, the method may further include: performing image enhancement and mean filtering preprocessing on the initial reference image.

[0027] Step S220: Determine the first frame of a series of consecutive frames as the reference image.

[0028] In practical applications, the volume of liquid in transparent bottles used to hold different types of liquids is not fixed, and the volume of liquid in different batches of transparent bottles holding the same type of liquid may also vary. Therefore, using a standard, defect-free transparent bottle image as a reference image is inaccurate. This application uses the first frame image as the reference image, ensuring its accuracy. Furthermore, because the liquid moves relative to the transparent bottle being inspected during image acquisition, even if the reference image contains defects, their position will change in the next frame due to the liquid's movement. Trajectory clustering can be used to accurately capture this trajectory, thereby improving the accuracy of defect detection. This application uses first frame reference image differential processing, which is unaffected by fixture vibration or slow changes in lighting, resulting in high consistency in defect extraction.

[0029] Step S230: Based on the reference image, perform defect analysis on each non-reference image to obtain the defect analysis results corresponding to each non-reference image.

[0030] Among them, the defect analysis results corresponding to the non-reference image may include: each target defect region and the defect center coordinates and defect features of each target defect region; the defect center coordinates may include the center row coordinates and the center column coordinates; the defect features may include: defect area, gray energy, aspect ratio and compactness.

[0031] In practice, for any non-reference image, the grayscale difference response, gradient difference response and local texture difference response of each pixel in the non-reference image and the corresponding pixel in the reference image are calculated respectively. The grayscale difference response, gradient difference response, and local texture difference response of each pixel are weighted and fused to obtain the fused differential response of each pixel; Based on the fusion difference response of each pixel, a fusion difference response map corresponding to the non-reference image is generated; Defect analysis is performed on the fused differential response map corresponding to the non-reference image to obtain the defect analysis results. Specifically, for any differential image, the region in the fused differential response map where the fused differential response is greater than a preset response threshold is identified as the first defect region. The preset response threshold can range from 0 to 245. The first defect region is a suspected defect region. Morphological operations are performed on the identified first defect regions to obtain multiple second defect regions. Connectivity analysis is performed on the multiple second defect regions to obtain multiple third defect regions. The third defect region corresponding to the defect area greater than a preset area threshold is identified as the target defect region of the non-reference image. The third defect region corresponding to the defect area greater than the preset area threshold is identified as minor noise and directly deleted. The defect area is the pixel area within each target defect region, i.e., the total number of pixels contained within each target defect region. Defect features of each target defect region are extracted, and a defect energy weight model is constructed. Based on the defect energy weight model, the defect center coordinates of each target defect region are calculated. The target defect regions of the non-reference image, along with their defect center coordinates and defect features, are used as the defect analysis results corresponding to the non-reference image.

[0032] In the above embodiments of this application, calculating the grayscale difference response, gradient difference response, and local texture difference response of each pixel in the non-reference image and the corresponding pixel in the reference image may include: The grayscale value and gradient magnitude of each pixel in the non-reference image and each pixel in the reference image are calculated respectively. Local texture features are extracted from each pixel in the non-reference image and each pixel in the reference image to obtain the texture features of each pixel. The Sobel operator can be used to calculate the gradient magnitude and the Local Binary Pattern (LBP) encoding can be used to extract the local texture features. For any pixel location, the difference between the gray value of that pixel location in the non-reference image and the gray value of that pixel location in the reference image is taken as the gray-level difference response at that pixel location, i.e., the gray-level difference response between the pixel at that pixel location in the non-reference image and the pixel at that pixel location in the reference image. The difference between the gradient magnitude at the pixel location in the non-reference image and the gradient magnitude at the pixel location in the reference image is taken as the gradient difference response at the pixel location, i.e., the gradient difference response between the pixel at the pixel location in the non-reference image and the pixel at the pixel location in the reference image. The difference between the texture features at the pixel location in the non-reference image and the texture features at the pixel location in the reference image is taken as the local texture difference response at the pixel location, i.e., the local texture difference response between the pixel at the pixel location in the non-reference image and the pixel at the pixel location in the reference image.

[0033] In the above embodiments of this application, weighted fusion of the grayscale difference response, gradient difference response, and local texture difference response of each pixel to obtain the fused differential response of each pixel may include: Calculate the overall brightness and local contrast of the non-reference image; where the overall brightness can be the mean gray level of the entire non-reference image; and the local contrast value can be the standard deviation of the gray level of the entire non-reference image. Based on the overall brightness and local contrast, and according to the preset weighting adjustment rules, the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient corresponding to the grayscale difference response, the gradient difference response, and the local texture difference response are determined respectively. Specifically, when the overall brightness value is high, the weighting coefficient of the grayscale difference response is increased; when the local contrast value is low, the weighting coefficients of the gradient difference response and the local texture difference response are increased; the sum of the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient is 1. The grayscale difference response, gradient difference response, and local texture difference response of each pixel are weighted and fused using the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient, respectively, to obtain the fused differential response of each pixel. Specifically, for any pixel location, the grayscale response at that pixel location is multiplied by the first weighting coefficient to obtain the first weighted value; the gradient difference response at that pixel location is multiplied by the second weighting coefficient to obtain the second weighted value; the texture feature at that pixel location is multiplied by the third weighting coefficient to obtain the third weighted value; the first weighted value, the second weighted value, and the third weighted value are added together to obtain the fused differential response of that pixel location; based on the fused differential response of each pixel location, the fused differential response map corresponding to the non-reference image is obtained.

[0034] In another embodiment of this application, performing morphological operations on a plurality of determined first defect regions to obtain a plurality of second defect regions may include: A rectangular structuring element is used to perform a morphological opening operation (erosion followed by dilation) on multiple determined first defect regions to obtain multiple first defect regions after the opening operation; the size of the structuring element can be OpenRow×OpenCol (default 2×2); by performing a morphological opening operation on multiple first defect regions, small, isolated, and discrete noise points, burrs, and pseudo-defects are removed, while the overall outline of the real defect is preserved and defect breakage is avoided; After the opening operation, a morphological closing operation (dilation followed by erosion) is performed on multiple first defect regions to obtain multiple second defect regions. The structuring element size can be CloseRow×CloseCol (default 3×3). By performing a morphological closing operation on multiple first defect regions after the opening operation, small holes and gaps inside the defects are filled. The defect edges that are broken due to difference or noise are connected to make the defect regions completely connected.

[0035] In another embodiment of this application, performing connected component analysis on multiple second defect regions to divide the interconnected set of pixels into independent individual defect targets, thereby obtaining multiple third defect regions, may include: The 8-neighborhood connectivity rule is used to traverse all pixels within each second defect region to determine the connectivity between pixels; each group of connected pixels is marked as an independent defect region, forming a non-overlapping set of defect candidates, resulting in multiple third defect regions.

[0036] In the above embodiments of this application, the defect features of each target defect region are extracted, and a defect energy weight model is constructed; based on the defect energy weight model, the defect center coordinates of each target defect region are calculated, which may include: For any target defect region, count the number of pixels contained in the target defect region to obtain the defect area of ​​the target defect region; The sum of squares of the fused differential responses of all pixels within the target defect region is taken as the grayscale energy of the target defect region. The ratio of the major axis to the minor axis of the smallest bounding rectangle containing the target defect region is taken as the aspect ratio of the target defect region. Extract the outline of the target defect region, calculate the perimeter of the outline, and calculate the compactness of the target defect region based on the defect area and perimeter. Specifically, extract the outline of the target defect region, obtain the coordinates of each outline point on the outline, calculate the Euclidean distance between each pair of adjacent outline points in turn, accumulate the Euclidean distances between all adjacent outline points, and use the accumulated result as the perimeter of the outline. The defect area, grayscale energy, aspect ratio, and compactness are used as the defect features of the target defect region. Based on preset grayscale energy weighting coefficients, area weighting coefficients, aspect ratio weighting coefficients, and compactness weighting coefficients, the defect area, grayscale energy, aspect ratio, and compactness of the target defect region are weighted and summed to obtain the defect energy weighting model of the target defect region. The defect energy weighting model is used to characterize the confidence level that the target defect region belongs to a real defect. The expression of the defect energy weighting model is as follows: W = k1 × E + k2 × A + k3 × (1 / R) + k4 × C; where E represents grayscale energy, A represents defect area, R represents aspect ratio, C represents compactness, W represents the defect energy weighting model of the target defect region, and k1, k2, k3, and k4 represent the grayscale energy weighting coefficient, area weighting coefficient, aspect ratio weighting coefficient, and compactness weighting coefficient, respectively. Obtain the pixel coordinates and fusion differential response of each pixel within the defective region of the target; The pixel weights of each pixel are determined based on the fusion difference response of each pixel. Based on the pixel weight and pixel coordinates of each pixel, the weighted average coordinates are calculated to obtain the center coordinates of the defect in the target defect area.

[0037] In another embodiment of this application, the method for determining the difference image corresponding to any non-reference image may further include: The non-reference image and the reference image are subtracted to obtain the initial difference image corresponding to the non-reference image. Specifically, point-to-point gray-level subtraction is used to subtract the non-reference image and the reference image, as shown in the following formula: ImageSub(x,y)=BaseReducedROI(x,y) CurImageROI(x,y); where ImageSub(x,y) represents the difference image corresponding to any non-reference image; BaseReducedROI(x,y) represents the reference image; CurImageROI(x,y) represents the non-reference image; The grayscale difference of each pixel in the initial difference image is mapped to a preset grayscale range to obtain the difference image corresponding to the non-reference image. Specifically, based on a preset scaling factor and a preset grayscale offset value, the grayscale difference of each pixel in the initial difference image is mapped to a preset grayscale range. The preset scaling factor can be 1, the preset grayscale offset value can be 255, and the preset grayscale range can be 0~255 to make the defective area appear as a bright development. The operation only processes pixels within the non-reference image, ignoring invalid areas outside the bottle, thus improving computational efficiency and anti-interference capabilities.

[0038] In practical applications, static backgrounds, dust, light spots, and uniform lighting changes have similar gray levels in two frames, with the difference approaching 0, and are therefore suppressed; while foreign objects or stains that rotate with the bottle have significant gray level differences in two frames, and the difference is amplified, forming a bright area.

[0039] In another embodiment of this application, the method for calculating the coordinates of the defect center of the target defect region may further include: The center coordinates of the target defect region are equal to the average of the row and column coordinates of all pixels in the region, i.e., the centroid of the region / first moment. Let there be N pixels in the defect region, with coordinates of (x1, y1), (x2, y2), (x3, y3)...(x...). N ,y N Then: the center row coordinates Row = (y1 + y2 + ... + y N ) / N; Center column coordinates Column=(x1+x2+…+x N Row = (1 / N)Σy; equivalent to the first moment formula: Row = (1 / N)Σy i ; Column = (1 / N)Σx i .

[0040] Step S240: Based on the defect analysis results corresponding to each non-reference image, perform defect trajectory clustering to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

[0041] In practice, any non-reference image is used as the non-reference image of the current frame; Based on the defect center coordinates and defect features of each target defect region in the current frame non-reference image, and the defect center coordinates and defect features of each target defect region in the previous frame non-reference image, calculate the comprehensive matching degree between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image. Based on the overall matching degree, the matching relationship between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image is determined; Defect trajectory clustering is performed based on matching relationships to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

[0042] In the above embodiments of this application, calculating the comprehensive matching degree between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image may include: A pre-built rotational motion prediction model is used to predict the defect center coordinates of each target defect region in the previous non-reference image, thus obtaining the predicted center coordinates of each target defect region in the current non-reference image. In practical applications, since the vial rotates at a fixed angular velocity during detection, the motion trajectory of the actual defect in consecutive frames is continuous and predictable. A rotational motion model is constructed to predict the trajectory. k =(Row k ,Column k );V k =P k P k 1; P predict =P k +V k Among them, P k Row represents the coordinates of the defect center of any target defect region in the k-th non-reference image; k Represents the row coordinates of any target defect region in the k-th non-reference image; Column k P represents the column coordinates of any target defect region in the k-th non-reference image; k 1 represents the coordinates of the defect center in the same target defect region in the (k-1)th frame of the non-reference image; P predict Indicates the coordinates of the predicted center; V k This represents the motion vector of any target defect region from frame (k-1) to frame k. For any target defect region in the current frame non-reference image, the defect distance between the target defect region and each target defect region in the previous frame non-reference image is calculated based on the defect center coordinates of the target defect region and the defect center coordinates of each target defect region in the previous frame non-reference image. Specifically, the Euclidean pixel distance between the defect center coordinates of the target defect region and the defect center coordinates of each target defect region in the previous frame non-reference image is used as the corresponding defect distance. Based on the defect features of the target defect region and the defect features of each target defect region in the previous non-reference image, the feature difference between the target defect region and each target defect region in the previous non-reference image is determined. The feature difference can include a first feature difference and a second feature difference. Specifically, the grayscale energy, aspect ratio, and compactness of the target defect region in the current non-reference image are combined into a first feature vector; the grayscale energy, aspect ratio, and compactness of the target defect region in the previous non-reference image are combined into a second feature vector; the weighted Euclidean distance between the first and second feature vectors is calculated as the first feature difference between the two target defect regions; the area difference between the defect area of ​​the target defect region in the current non-reference image and the defect area of ​​the target defect region in the previous non-reference image is calculated to obtain the second feature difference. Calculate the difference between the coordinates of each predicted center and the coordinates of the defect center of each target defect region to obtain the predicted position deviation between the target defect region and each target defect region in the previous frame non-reference image; the predicted position deviation includes the deviation in the row coordinate direction and the deviation in the column coordinate direction. Based on the defect distance, feature difference, and predicted position deviation, the comprehensive matching degree between the target defect region and each target defect region in the previous non-reference image is calculated. Specifically, preset defect distance weight coefficient, first feature difference weight coefficient, second feature difference weight coefficient, and predicted position deviation weight coefficient are obtained. Among them, the defect distance weight coefficient, first feature difference weight coefficient, second feature difference weight coefficient, and predicted position deviation weight coefficient are all positive numbers, and the sum of the four is equal to 1. The defect distance, first feature difference, second feature difference weight coefficient, and predicted position deviation are weighted and summed using the defect distance weight coefficient, first feature difference weight coefficient, second feature difference weight coefficient, and predicted position deviation weight coefficient to obtain the comprehensive matching degree between the target defect region and each target defect region in the previous non-reference image.

[0043] In the above embodiments of this application, defect trajectory clustering based on matching relationships to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images may include: Two target defect regions whose overall matching degree meets the preset matching conditions are determined to be successfully matched; specifically, the two target defect regions with the smallest overall matching degree and an overall matching degree less than the preset matching threshold are determined to be successfully matched. If any target defect region in the current frame of the non-reference image successfully matches the defect trajectory of any target defect region in the previous frame of the non-reference image, then the defect center coordinates of the corresponding target defect region in the current frame of the non-reference image are appended to the successfully matched defect trajectory in chronological order, and the trajectory numbers of the defect trajectories in consecutive frames are unified. Specifically, when it is determined that any target defect region in the current frame of the non-reference image successfully matches any target defect region in the previous frame of the non-reference image, it indicates that these two target defect regions belong to the same defect and are imaged at different times. At this time, the acquisition... The defect motion trajectory to which the matched target defect region belongs in the previous frame non-reference image is determined. This defect motion trajectory already contains the defect center coordinates of all target defect regions corresponding to the defect from the first frame to the previous frame. The defect center coordinates of the target defect region in the current frame non-reference image are appended to the end of the above defect motion trajectory in chronological order. Since the current frame is later than the previous frame, and the last coordinate point in the defect motion trajectory is the defect center coordinate of the target defect region in the previous frame, the coordinate points in the defect motion trajectory still maintain a strict chronological order after appending. If any target defect region in the current frame's non-reference image does not match the defect motion trajectory of any target defect region in the previous frame's non-reference image, a new defect motion trajectory is created for the corresponding target defect region in the current frame's non-reference image, and a unique trajectory number is assigned to each defect motion trajectory. Specifically, the defect center coordinates of the target defect region are used as the first coordinate point (i.e., the starting point) of this new defect motion trajectory, and a unique trajectory number is assigned to this new defect motion trajectory. Uniqueness means that this trajectory number is not repeated among all existing defect motion trajectories, and is used to distinguish different defect motion trajectories. The process iterates through all non-reference images, generating defect motion trajectories for each defect across multiple consecutive frames through frame-by-frame recursive matching. Specifically, the traversal order starts from the second non-reference image (i.e., the third frame in a series of consecutive images) and proceeds frame by frame until the last non-reference image is processed. The processing flow for each frame is the same: the current non-reference image is used as the reference image, and it is matched with the previous non-reference image. Based on the matching result, the image is either appended to an existing trajectory or a new trajectory is created. Through this frame-by-frame recursive matching method, the target defect region detected in each frame is assigned to an existing trajectory or a new trajectory. After traversing all frames, all target defect regions belonging to the same real defect are connected into a complete defect motion trajectory, thus generating the complete defect motion trajectory for each defect across multiple consecutive frames.

[0044] In another embodiment of this application, the method for determining the trajectory of the defect may further include: For any frame of non-reference image, the non-reference image and the previous frame of non-reference image are taken as a pair of adjacent frames; calculate the defect distance between the defect center coordinates in the adjacent frame pair; specifically, the Euclidean distance is used to calculate the defect distance between the defect center coordinates in the adjacent frame pair. If the defect distance is less than the preset defect distance threshold, the center coordinates of the two defects corresponding to the defect distance are determined as the position of the same defect in the adjacent frame pair; specifically, a temporal constraint of matching only adjacent frames and a spatial constraint of the distance being less than the preset defect distance threshold are set, and if the conditions are met, they are determined to be the same defect. Traverse all adjacent frame pairs and check whether any defect in the adjacent frame pair matches any defect that has been detected in the adjacent frame pairs preceding the current one. If the defect matches any detected defect, the position of the defect in the adjacent frame pair is added to the end of the defect motion trajectory of the matched defect in chronological order, generating the defect motion trajectory of the defect in multiple consecutive frames, and unifying the trajectory number of the defect motion trajectory in multiple consecutive frames. If the defect does not match any of the detected defects, the position of the defect in the adjacent frame pair is arranged in temporal order to generate the defect motion trajectory in multiple consecutive frames, and a unique trajectory number is assigned to the defect motion trajectory. By iteratively associating frame by frame, the discrete defect centers are connected in temporal order to complete the clustering of the entire sequence of defect trajectories and form multiple original defect motion trajectories.

[0045] In another embodiment of this application, after obtaining the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images, the method may further include: Based on the defect motion trajectory of any defect in the transparent bottle to be detected in multiple consecutive frames of images, calculate the characteristic fluctuation difference, motion direction, trajectory length and trajectory confidence score of the defect. If the trajectory length, direction of movement, trajectory confidence score, and feature fluctuation difference of a defect all meet the configured preset valid defect conditions, then the defect is determined to be a valid defect. Specifically, when the trajectory length is greater than a preset length threshold, the trajectory confidence score is greater than a preset confidence threshold, the feature fluctuation difference meets the preset fluctuation threshold condition, and the direction of movement conforms to the rotation prediction model, the configured preset valid defect conditions are met. Among them, the preset length threshold can be 3; the confidence threshold can be 0.6; the preset fluctuation threshold conditions can include: defect area fluctuation threshold, grayscale energy fluctuation threshold, aspect ratio fluctuation threshold, and compactness fluctuation threshold; when the difference in defect area fluctuation, grayscale energy fluctuation, aspect ratio fluctuation, and compactness fluctuation are all less than their respective preset fluctuation thresholds, the preset fluctuation threshold conditions are met; the direction of movement can include two dimensions: rotation angle and direction of movement; if the direction of movement falls within the preset standard direction of movement range, the direction of movement is determined to conform to the rotation prediction model. If the trajectory length, direction of movement, trajectory reliability score, or feature fluctuation difference of the defect does not meet the preset valid defect conditions, the defect will be determined as an invalid defect. Based on the defect motion trajectory of each effective defect in multiple consecutive frames of images, the target defect detection result of the transparent bottle to be detected is obtained; wherein, the target defect detection result may include: the defect motion trajectory of each effective defect in multiple consecutive frames of images, the defect center coordinates, the trajectory number and the trajectory length.

[0046] In the above embodiments of this application, calculating the characteristic fluctuation difference, movement direction, trajectory length, and trajectory reliability score of any defect in the transparent bottle to be detected based on the defect motion trajectory in multiple consecutive frames of images may include: The total number of defect center coordinates included in the defect's trajectory is counted to obtain the trajectory length; For any two adjacent defect center coordinates in the defect's motion trajectory, calculate the inter-frame displacement vector of the two adjacent defect center coordinates; calculate the rotation angle of the defect based on the angle change of the inter-frame displacement vector; determine the movement direction of the defect based on the directional change trend of each inter-frame displacement vector; and determine the direction of motion based on the rotation angle and the movement direction. Obtain the defect features of the target defect region corresponding to the center coordinates of each defect in the defect's movement trajectory; calculate the feature change of the defect feature corresponding to any two adjacent center coordinates of the defect in the defect's movement trajectory, and normalize the feature change to obtain the normalized fluctuation difference of the defect feature between any two adjacent center coordinates of the defect; sum or take the maximum value of the normalized fluctuation difference of the same defect feature between any two adjacent center coordinates of the defect to obtain the defect area fluctuation difference, grayscale energy fluctuation difference, aspect ratio fluctuation difference, and compactness fluctuation difference of the defect; use the defect area fluctuation difference, grayscale energy fluctuation difference, aspect ratio fluctuation difference, and compactness fluctuation difference of the defect as the defect feature fluctuation difference of the defect. The number of consecutive matches is obtained by counting the number of defect center coordinates in the defect's motion trajectory. The number of consecutive matches represents the number of times adjacent frames in the defect's motion trajectory are successfully matched. Since the defect's motion trajectory is generated by recursively matching frame by frame, each defect center coordinate in the defect's motion trajectory represents the position of the defect in a certain frame. Any two adjacent defect center coordinates in the trajectory have been determined to be a successful match during the trajectory clustering process. Therefore, for a defect's motion trajectory containing N defect center coordinates, the number of its adjacent frame pairs is N-1, which is the number of consecutive matches. Based on the defect center coordinates, characteristic fluctuation differences, and movement direction of the defect in its motion trajectory, a trajectory continuity score is calculated. Specifically, a centroid displacement sequence is generated based on the displacement of the defect center coordinates in any adjacent frame in the defect motion trajectory; a characteristic fluctuation sequence is constructed based on the characteristic changes of the defect features corresponding to any two adjacent defect center coordinates in the defect motion trajectory; the deviation between the direction angle and the movement direction of the displacement vector between each frame is calculated to obtain the degree of direction deviation; and the dispersion of the centroid displacement sequence, movement direction, and characteristic fluctuation sequence is analyzed to obtain the trajectory continuity score of the defect motion trajectory. Based on the defect center coordinates and the corresponding predicted center coordinates in the defect movement trajectory, a motion consistency score is calculated. Specifically, the positional deviation between the defect center coordinates and the corresponding predicted center coordinates is calculated. The motion consistency score is determined based on the average value of each positional deviation. Alternatively, the average value of each positional deviation can be mapped to obtain the motion consistency score according to a preset mapping rule; or, the reciprocal of the average value of each positional deviation can be used as the motion consistency score; or, a preset positional deviation threshold can be obtained, the average value of each positional deviation can be normalized, and the normalized value can be subtracted from one to obtain the motion consistency score. The trajectory credibility score is calculated based on the number of consecutive matches, the trajectory continuity score, and the motion consistency score. Specifically, the trajectory credibility score is obtained by weighted summation of the number of consecutive matches, the trajectory continuity score, and the motion consistency score based on preset first coefficient, preset second coefficient, and preset third coefficient.

[0047] This application identifies true defects by using continuous trajectories. Static noise, random light spots, and dust cannot form continuous trajectories and can be directly filtered out, significantly reducing the false detection rate.

[0048] Corresponding to the above method, embodiments of this application also provide a defect visual tracking device, such as... Figure 3 As shown, the device includes: The acquisition unit 310 is used to acquire multiple consecutive frames of images of the transparent bottle to be detected when the liquid filled in the transparent bottle to be detected moves relative to the transparent bottle to be detected. The determining unit 320 is used to determine the first frame image in a series of consecutive frames as the reference image; Analysis unit 330 is used to perform defect analysis on each non-baseline image based on the baseline image, and obtain the defect analysis results corresponding to each non-baseline image; Clustering unit 340 is used to cluster defect trajectories based on the defect analysis results corresponding to each non-reference image, so as to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

[0049] The functions of each functional unit of the defect visual tracking device provided in the above embodiments of this application can be implemented through the above methods and steps. Therefore, the specific working process and beneficial effects of each unit in the defect visual tracking device provided in the embodiments of this application will not be repeated here.

[0050] This application also provides an electronic device, such as... Figure 4 As shown, it includes a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440.

[0051] Memory 430 is used to store computer programs; When the processor 410 executes the program stored in the memory 430, it performs the following steps: While the liquid filling the transparent bottle to be tested is moving relative to the transparent bottle to be tested, acquire a series of multiple frames of images of the transparent bottle to be tested. The first frame in a series of consecutive images is determined as the reference image; Based on the baseline image, defect analysis is performed on each non-baseline image to obtain the defect analysis results corresponding to each non-baseline image. Based on the defect analysis results corresponding to each non-reference image, defect trajectory clustering is performed to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

[0052] The communication bus mentioned above can be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0053] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0054] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0055] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0056] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments for solving the problem can be found in [reference needed]. Figure 2 The steps in the illustrated embodiments are used to implement the electronic device. Therefore, the specific working process and beneficial effects of the electronic device provided in this application will not be repeated here.

[0057] Those skilled in the art will understand that the embodiments in this application can be provided as methods, systems, or computer program products. Therefore, the embodiments in this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects.

[0058] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0059] Although preferred embodiments have been described in this application, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.

[0060] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of this application and its equivalents, then these modifications and variations are also intended to be included in the embodiments of this application.

Claims

1. A defect visual tracking method, characterized in that, The method includes: While the liquid filling the transparent bottle to be tested moves relative to the transparent bottle to be tested, acquire a series of multiple frames of images of the transparent bottle to be tested. The first frame in a series of consecutive images is determined as the reference image; Based on the reference image, defect analysis is performed on each non-reference image to obtain the defect analysis results corresponding to each non-reference image. Based on the defect analysis results corresponding to each non-reference image, defect trajectory clustering is performed to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

2. The method as described in claim 1, characterized in that, While the liquid filling the transparent bottle to be tested moves relative to the transparent bottle, a series of multiple frames of images of the transparent bottle to be tested are acquired, including: While the liquid moves relative to the transparent bottle to be detected, acquire multiple consecutive initial images of the transparent bottle to be detected; The first frame of a series of consecutive initial images is used as the initial reference image. Based on the initial reference image, a caliper measurement model is constructed; wherein, the caliper measurement model is used to locate the edges of the initial reference image; Based on the size parameters of the transparent bottle to be tested and the caliper measurement model, the vertex coordinates of the region to be extracted are determined; Based on the vertex coordinates of the region to be extracted, the initial images of multiple consecutive frames are cropped to obtain multiple consecutive frames of images.

3. The method as described in claim 1, characterized in that, Based on the reference image, defect analysis is performed on each non-reference image to obtain the defect analysis results for each non-reference image, including: For any non-reference image, calculate the grayscale difference response, gradient difference response, and local texture difference response of each pixel in the non-reference image and the corresponding pixel in the reference image; The grayscale difference response, gradient difference response, and local texture difference response of each pixel are weighted and fused to obtain the fused differential response of each pixel; Based on the fusion difference response of each pixel, a fusion difference response map corresponding to the non-reference image is generated; Defect analysis is performed on the fused differential response map corresponding to the non-reference image to obtain the defect analysis results corresponding to the non-reference image.

4. The method as described in claim 3, characterized in that, Defect analysis is performed on the fused differential response map corresponding to the non-reference image to obtain the defect analysis results corresponding to the non-reference image, including: For any fused differential response map, the region in the fused differential response map where the fused differential response is greater than a preset response threshold is determined as the first defect region; Morphological operations are performed on multiple identified first defect regions to obtain multiple second defect regions; By performing connected component analysis on multiple second defect regions, multiple third defect regions are obtained; The third defect region corresponding to the defect area that is larger than the preset area threshold is determined as the target defect region of the non-reference image. Extract the defect features of each target defect region and construct a defect energy weight model; Based on the defect energy weighting model, calculate the defect center coordinates of each target defect region; The target defect regions of the non-reference image, along with the defect center coordinates and defect features of each target defect region, are used as the defect analysis results corresponding to the non-reference image.

5. The method as described in claim 1, characterized in that, The defect analysis results include each target defect region in the non-reference image, as well as the defect center coordinates and defect features of each target defect region; Based on the defect analysis results corresponding to each non-reference image, defect trajectory clustering is performed to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images, including: Use any non-reference image as the non-reference image of the current frame; Based on the defect center coordinates and defect features of each target defect region in the current frame non-reference image and the defect center coordinates and defect features of each target defect region in the previous frame non-reference image, calculate the comprehensive matching degree between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image. Based on the comprehensive matching degree, the matching relationship between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image is determined; Based on the matching relationship, defect trajectory clustering is performed to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

6. The method as described in claim 5, characterized in that, Calculating the overall matching degree between each target defect region in the current frame non-reference image and each target defect region in the previous frame non-reference image includes: The defect center coordinates of each target defect region in the previous non-reference image are predicted by using a pre-built rotational motion prediction model, and the predicted center coordinates of each target defect region in the previous non-reference image are obtained in the current non-reference image. For any target defect region in the current frame non-reference image, the defect distance between the target defect region and each target defect region in the previous frame non-reference image is calculated based on the defect center coordinates of the target defect region and the defect center coordinates of each target defect region in the previous frame non-reference image. Based on the defect features of the target defect region and the defect features of each target defect region in the previous non-reference image, the feature difference between the target defect region and each target defect region in the previous non-reference image is determined. Calculate the difference between the predicted center coordinates and the defect center coordinates of each target defect region to obtain the predicted position deviation between the target defect region and each target defect region in the previous non-reference image. Based on the defect distance, the feature difference, and the predicted position deviation, the comprehensive matching degree between the target defect region and each target defect region in the previous frame non-reference image is calculated.

7. The method as described in claim 5, characterized in that, Based on the matching relationship, defect trajectory clustering is performed to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images, including: Two target defect regions whose overall matching degree meets the preset matching conditions have been successfully matched. If any target defect region in the current frame non-reference image successfully matches the defect motion trajectory to which any target defect region in the previous frame non-reference image belongs, then the defect center coordinates of the corresponding target defect region in the current frame non-reference image are appended to the successfully matched defect motion trajectory in chronological order, and the trajectory number of the defect motion trajectory in the consecutive frames of images is unified. If any target defect region in the current frame non-reference image fails to match the defect motion trajectory to which any target defect region in the previous frame non-reference image belongs, then a new defect motion trajectory is created for the corresponding target defect region in the current frame non-reference image, and a unique trajectory number is assigned to each defect motion trajectory. By iterating through all non-reference images and performing frame-by-frame recursive matching, the defect motion trajectory of each defect in consecutive multi-frame images is generated.

8. The method as described in claim 6, characterized in that, After obtaining the defect trajectory of the transparent bottle to be detected in multiple consecutive frames of images, the method further includes: Based on the defect motion trajectory and defect features of any defect in the transparent bottle to be detected in multiple consecutive frames of images, calculate the feature fluctuation difference, motion direction, trajectory length and trajectory reliability score of the defect; If the trajectory length, direction of movement, trajectory reliability score, and feature fluctuation difference of the defect all meet the configured preset valid defect conditions, then the defect is determined to be a valid defect. If the trajectory length, direction of movement, trajectory reliability score, or feature fluctuation difference of the defect does not meet the preset valid defect conditions, the defect will be determined as an invalid defect. Based on the defect center coordinates, defect motion trajectory, trajectory number, and trajectory length of each effective defect in multiple consecutive frames of images, the target defect detection results of the transparent bottle to be detected are obtained.

9. The method as described in claim 8, characterized in that, Based on the defect trajectory and features of any defect in the transparent bottle to be detected across multiple consecutive frames of images, a trajectory reliability score for the defect is calculated, including: The number of consecutive matches is obtained based on the number of defect center coordinates contained in the defect's motion trajectory. Calculate the trajectory continuity score based on the center coordinates of each defect, the characteristic fluctuation difference, and the direction of movement of each defect in the defect's trajectory. Based on the center coordinates of each defect in the defect's motion trajectory and the predicted center coordinates corresponding to each defect center coordinate, calculate the motion consistency score; The trajectory credibility score is calculated based on the number of consecutive matches, the trajectory continuity score, and the motion consistency score.

10. A defect visual tracking device, characterized in that, The device includes: The acquisition unit is used to acquire multiple consecutive frames of images of the transparent bottle to be detected when the liquid filled in the transparent bottle to be detected moves relative to the transparent bottle to be detected. The determining unit is used to determine the first frame of a series of consecutive frames as the reference image; The analysis unit is used to perform defect analysis on each non-reference image based on the reference image, and obtain the defect analysis results corresponding to each non-reference image. The clustering unit is used to cluster defect trajectories based on the defect analysis results corresponding to each non-reference image, so as to obtain the defect motion trajectory of the transparent bottle to be detected in multiple consecutive frames of images.

Images