New energy vehicle seal grasping method and system based on machine vision

Abstract

The application provides a new energy vehicle sealing piece grabbing method and system based on machine vision, a two-dimensional edge curve set and a spatial height distribution set of a sealing piece are constructed through a real-time surface image of the sealing piece; a curvature fluctuation parameter of a local bending state of the sealing piece is extracted according to the two-dimensional edge curve set, and a visual deformation field of the corresponding sealing piece is generated; a stable support area and an elastic rebound area of the sealing piece are divided according to the visual deformation field, and an evaluation model of grabbing stability is constructed according to the stable support area; the evaluation model is matched and analyzed with a current clamping direction of a mechanical gripper, and then an offset compensation amount between the clamping direction and a local deformation direction is determined with the elastic rebound area; a grabbing compensation pose of the end of a mechanical arm is generated based on the offset compensation amount, and the mechanical arm is controlled to move to a target grabbing area. By adopting the scheme, dynamic compensation control of the grabbing direction can be realized in the conveying scene of the deformation rebound of the flexible sealing piece of the new energy vehicle.

Description

Technical Field

[0001] This application relates to the field of robotic arm grasping and control technology, and more specifically, to a method and system for grasping new energy vehicle seals based on machine vision. Background Technology

[0002] The gripping of flexible seals for new energy vehicles refers to the process of positioning, picking up, and placing rubber or elastomer seals continuously conveyed on the conveyor line using the grippers at the end of a robotic arm on an automated production line. Its goal is to achieve high-speed, stable, and non-destructive automated gripping to meet the requirements of large-scale manufacturing of new energy vehicles for sealing assembly efficiency and yield.

[0003] However, existing technologies, such as traditional gripping methods, typically rely on fixed trajectory teaching or visual positioning based on simple edge detection. These methods struggle to handle real-time local bending, stacking, or random posture changes in flexible seals used in new energy vehicles on conveyor lines. Due to the high elasticity of the seals, unstable deformation zones and rebound tendencies form on their surfaces during transport. Existing methods cannot dynamically perceive the continuity of deformation on the seal surface, nor can they quantify the coupling relationship between curvature fluctuations and spatial height distribution. This often leads to the grippers incorrectly selecting areas prone to slippage or severe rebound, causing gripping deviation, dropping, or even damage to the seals. Therefore, achieving dynamic compensation control of the gripping direction to reduce gripping deviation caused by flexible rebound in conveying scenarios where flexible seals in new energy vehicles deform and rebound has become a significant challenge for the industry. Summary of the Invention

[0004] This application provides a machine vision-based method and system for grasping seals in new energy vehicles, which can achieve dynamic compensation control of the grasping direction in the conveying scenario where the flexible seals of new energy vehicles deform and rebound.

[0005] In a first aspect, this application provides a machine vision-based method for grasping sealing components of new energy vehicles, comprising the following steps: Real-time surface images of flexible seals for new energy vehicles in the conveyor line are obtained, and a set of two-dimensional edge curves and a set of spatial height distributions of the corresponding seals are constructed based on the real-time surface images. Based on the curvature variation relationship between adjacent edge segments in the set of two-dimensional edge curves, the curvature fluctuation parameters of the local bending state of the seal are extracted, and the visual deformation field of the corresponding seal is generated based on the curvature fluctuation parameters and the set of spatial height distribution. Based on the degree of deformation continuity in different regions of the visual deformation field, the stable support region and the elastic rebound region of the seal are divided, and an evaluation model of gripping stability is constructed based on the boundary continuity of the stable support region. The evaluation model for gripping stability is matched and analyzed with the current gripping direction of the mechanical gripper. Then, based on the matching analysis results and the elastic rebound area, the offset compensation amount between the gripping direction and the local deformation direction is determined. Based on the offset compensation amount, the robotic arm end effector is generated to obtain the grasping compensation pose, and the robotic arm is controlled to move to the target grasping area.

[0006] In conjunction with the first aspect, in one possible implementation, extracting the curvature fluctuation parameters of the local bending state of the seal based on the curvature variation relationship of adjacent edge segments in the set of two-dimensional edge curves specifically includes: Obtain the discrete curvature sequence of each curve segment in the set of two-dimensional edge curves; Traverse all connection points of adjacent curve segments, and calculate the absolute difference of curvature of each connection point on the two adjacent curve segments based on the discrete curvature sequence to obtain a set of curvature difference values. All curvature differences in the curvature difference set are summarized according to the corresponding connection point position and difference amplitude to generate curvature fluctuation parameters of the local bending state of the seal.

[0007] In conjunction with the first aspect, in one possible implementation, generating the visual deformation field of the corresponding seal based on the curvature fluctuation parameters and the spatial height distribution set specifically includes: Based on the fluctuation intensity level in the curvature fluctuation parameters, assign a corresponding deformation energy weight to each connection point location; The deformation energy weights are mapped to the corresponding coordinate regions of the spatial height distribution set according to the connection point positions, and the relative height values ​​within the region are incrementally superimposed. The superimposed spatial height distribution set is used as the visual deformation field of the seal.

[0008] In conjunction with the first aspect, in one possible implementation, dividing the stable support region and elastic rebound region of the seal according to the degree of deformation continuity in different regions of the visual deformation field specifically includes: Calculate the standard deviation of the relative height value within each local window in the visual deformation field, and use it as an index of the deformation continuity of that window; All local windows with a deformation continuity index higher than a preset threshold are merged into the stable support area of ​​the seal, and all local windows with a deformation continuity index lower than a preset threshold are merged into the elastic rebound area of ​​the seal. Output a binary mask for the stable support region and a binary mask for the elastic rebound region.

[0009] In conjunction with the first aspect, in one possible implementation, constructing an evaluation model for grasping stability based on the boundary continuity of the stable support region specifically includes: Extract the sequence of outer boundary contour points from the binary mask of the stable support region; Determine the boundary continuity index of the outer boundary contour point sequence; The boundary continuity index and the area parameter of the stable support region are weighted and fused to construct an evaluation model for outputting the grasping stability.

[0010] In conjunction with the first aspect, in one possible implementation, the evaluation model for gripping stability is matched and analyzed with the current gripping direction of the mechanical gripper. Then, based on the matching analysis results and the elastic rebound region, the offset compensation amount between the gripping direction and the local deformation direction is determined. Specifically, this includes: The main direction of the stable support region is extracted from the evaluation model of gripping stability and used as the recommended gripping direction; Obtain the current gripping direction of the mechanical gripper, calculate its angular deviation from the recommended gripping direction, and obtain the direction matching analysis result; Based on the direction matching analysis results and the main direction of local deformation within the elastic rebound area, the offset compensation amount between the clamping direction and the local deformation direction is determined.

[0011] In conjunction with the first aspect, in one possible implementation, generating the grasping compensation pose of the robotic arm end effector based on the offset compensation amount and controlling the robotic arm to move to the target grasping area specifically includes: Obtain the angle offset compensation amount and displacement offset compensation amount from the offset compensation amount, and combine them with the current end-effector pose to generate the grasping compensation pose of the end-effector. Calculate the geometric center of the stable support region based on the binary mask, and use the geometric center as the target grasping region. The grasping compensation pose is converted into robotic arm joint motion commands, driving the robotic arm end effector to move to the target grasping area.

[0012] Secondly, this application provides a machine vision-based system for grasping sealing components of new energy vehicles, comprising: The acquisition module is used to acquire real-time surface images of flexible seals for new energy vehicles in the conveyor line, and to construct a set of two-dimensional edge curves and a set of spatial height distributions of the corresponding seals based on the real-time surface images. The processing module is used to extract the curvature fluctuation parameters of the local bending state of the seal based on the curvature change relationship of adjacent edge segments in the set of two-dimensional edge curves, and generate the visual deformation field of the corresponding seal based on the curvature fluctuation parameters and the set of spatial height distribution. The processing module is also used to divide the stable support region and the elastic rebound region of the seal according to the degree of deformation continuity of different regions in the visual deformation field, and to construct an evaluation model of gripping stability based on the boundary continuity of the stable support region. The processing module is also used to perform a matching analysis between the evaluation model of the gripping stability and the current gripping direction of the mechanical gripper, and then determine the offset compensation amount between the gripping direction and the local deformation direction based on the matching analysis result and the elastic rebound area. The execution module is used to generate the grasping compensation pose of the robotic arm end based on the offset compensation amount, and control the robotic arm to move to the target grasping area.

[0013] Thirdly, this application provides a computer device, which includes a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-described machine vision-based method for grasping new energy vehicle seals.

[0014] Fourthly, this application provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the aforementioned machine vision-based method for grasping seals of new energy vehicles.

[0015] The technical solution provided in this application has the following beneficial effects: This application's solution, firstly, by fusing curvature fluctuation parameters and spatial height distribution, constructs a visual deformation field that quantifies the energy distribution and three-dimensional morphological distortion caused by local bending of the seal. This enables the system to dynamically perceive the spatial distribution of flexible seal deformation for the first time on the conveyor line, providing a precise deformation data foundation for subsequent real-time differentiation between stable and rebound regions, thereby supporting dynamic prediction and compensation control of the gripping direction. Secondly, based on the deformation continuity of the visual deformation field, this application automatically divides the stable support region and the elastic rebound region, and constructs a gripping stability evaluation model based on boundary continuity. This model can quantify the gripping suitability of each region, effectively identify weak points prone to elastic rebound, and indicate the regions to be avoided and targeted for directional compensation control, fundamentally preventing gripping failures caused by selecting the wrong gripping position. Then, the gripping stability evaluation model is matched with the current direction of the mechanical gripper, and combined with the elastic rebound region... Based on the main direction of local deformation within the seal, this application can calculate the angle and displacement compensation between the clamping direction and the local deformation direction, enabling quantitative prediction of the seal's rebound trend and generation of compensation parameters. This allows the gripping action to actively adapt to the material deformation law rather than passively resisting rebound, improving the robustness of dynamic gripping. Finally, based on the offset compensation amount, the gripping compensation pose of the robotic arm end is generated, and the visual deformation field is updated in real time during movement to fine-tune the compensation amount in a closed loop. This application achieves full-process dynamic adaptive control of the gripping direction. Even if the seal continuously deforms or rebounds on the conveyor line, the robotic arm can correct the end posture in real time, always aligning the gripper with the optimal gripping area. This overcomes the problem that traditional fixed-track gripping is prone to missing the target or damaging the workpiece in flexible seal scenarios. In summary, this solution can achieve dynamic compensation control of the gripping direction in the conveying scenario where flexible seals in new energy vehicles deform and rebound, thereby reducing gripping offset caused by flexible rebound. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is an exemplary flowchart of a machine vision-based method for grasping seals of new energy vehicles, according to some embodiments of this application. Figure 2 This is an exemplary flowchart illustrating the determination of curvature fluctuation parameters according to some embodiments of this application; Figure 3 This is an exemplary flowchart illustrating the construction of an evaluation model for crawl stability according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of a machine vision-based new energy vehicle seal gripping system according to some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device for implementing a machine vision-based method for grasping seals of new energy vehicles, according to some embodiments of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] refer to Figure 1 The figure is an exemplary flowchart of a machine vision-based method for grasping seals of new energy vehicles, according to some embodiments of this application. The machine vision-based method for grasping seals of new energy vehicles mainly includes the following steps: In step 101, a real-time surface image of the flexible seal of the new energy vehicle in the conveyor line is obtained, and a two-dimensional edge curve set and a spatial height distribution set of the corresponding seal are constructed based on the real-time surface image.

[0020] In specific implementation, acquiring real-time surface images of the flexible sealing components of new energy vehicles in the conveyor line can be achieved in the following way: three high-speed industrial cameras are installed directly above and on both sides of the conveyor line. Polarizing filters are added in front of the camera lenses to eliminate reflections on the rubber surface of the flexible sealing components. At the same time, a trigger module synchronized with the motion encoder of the conveyor line is configured. When the sealing component moves with the conveyor line to the center field of view of the camera, the encoder sends a pulse signal to trigger the three cameras to take pictures simultaneously. Each camera acquires an 8-bit grayscale image with a resolution of 1280×1024 pixels and an exposure time of 50 microseconds to avoid motion blur. The three acquired images are labeled as top view, left view, and right view, respectively. Motion compensation is then performed on the images based on the relationship between the conveyor line speed and the camera frame rate, that is, the images are cropped or translated according to the displacement between adjacent frames. Finally, a set of real-time surface images that have been de-reflected, de-blurred, and time-aligned are output. Other methods can also be used in other embodiments, which are not specifically limited here.

[0021] It should be noted that the real-time surface image in this application refers to a set of multi-view grayscale images used to characterize the surface texture and geometric contour information of the flexible seal of the new energy vehicle at the current moment on the conveyor line. It can reflect the true appearance of the seal in a state without reflective interference, motion blur removed and time aligned.

[0022] In specific implementation, the construction of the two-dimensional edge curve set and spatial height distribution set of the corresponding seal based on the real-time surface image can be achieved in the following way: First, extract the top-view grayscale image from the real-time surface image, smooth and denoise it using Gaussian filtering, then extract the outer contour pixels of the seal using the Canny edge detection operator, and then connect the discrete edge points into an ordered edge chain using the 8-neighborhood connected component tracing algorithm. For each edge chain, perform Douglas-Pock segment approximation, and remove short noise segments with a length less than 10 pixels to obtain a two-dimensional edge curve set composed of several continuous curve segments; In this process, a stereo image pair is formed using the left and right views. After epipolar correction of the left and right views, a semi-global matching algorithm is used to calculate the disparity value of each pixel to generate a disparity map. Then, based on pre-calibrated camera parameters such as baseline distance and focal length, the disparity value is converted into a depth value. Using the transport line plane as the zero reference plane, the depth value is converted into a relative height value, thereby constructing a spatial height distribution set that corresponds one-to-one with the pixel position of the top view. Each element in this set records the pixel coordinates (x, y) and its corresponding relative height z. Other methods can also be used in other embodiments, which are not limited here.

[0023] It should be noted that the two-dimensional edge curve set in this application refers to a set of ordered curve segments that characterize the outer contour geometry of the flexible seal of a new energy vehicle on the conveyor line plane; the spatial height distribution set in this application refers to a three-dimensional data array that characterizes the height information of each point on the surface of the flexible seal of a new energy vehicle relative to the conveyor line plane, and each element in this set contains pixel coordinates and its corresponding relative height.

[0024] In step 102, based on the curvature change relationship of adjacent edge segments in the two-dimensional edge curve set, the curvature fluctuation parameters of the local bending state of the seal are extracted, and the visual deformation field of the corresponding seal is generated based on the curvature fluctuation parameters and the spatial height distribution set.

[0025] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flowchart for determining curvature fluctuation parameters in some embodiments of this application. In this embodiment, the extraction of curvature fluctuation parameters of the local bending state of the seal based on the curvature change relationship of adjacent edge segments in the two-dimensional edge curve set can be achieved by the following steps: In step 1021, the discrete curvature sequence of each curve segment in the two-dimensional edge curve set is obtained; In step 1022, all connection points of adjacent curve segments are traversed, and the absolute difference of curvature between each connection point on the two adjacent curve segments is calculated based on the discrete curvature sequence to obtain a set of curvature difference values. In step 1023, all curvature differences in the curvature difference set are summarized according to the corresponding connection point position and difference amplitude to generate curvature fluctuation parameters of the local bending state of the seal.

[0026] In specific implementation, the discrete curvature sequence of each curve segment in the two-dimensional edge curve set can be obtained in the following way: each curve segment in the two-dimensional edge curve set is regarded as a discrete point chain composed of several continuous pixels. For each curve segment, each pixel on it is traversed in order from the starting point to the ending point. The current pixel and its two adjacent pixels (the previous point and the next point) are taken. The approximate curvature value of the current pixel is calculated using the cosine theorem or by the cross product and dot product of vectors based on the triangle formed by these three points. For the two endpoints of the curve segment, the curvature values ​​of its two neighboring points are used as substitutes. Finally, each curve segment obtains a curvature value sequence equal to the number of pixels. The curvature value sequences of all curve segments are organized together in the original order of the curve segments to form the discrete curvature sequence.

[0027] In specific implementation, the connection points of all adjacent curve segments are traversed, and the absolute difference of curvature between each connection point on the two adjacent curve segments is calculated based on the discrete curvature sequence to obtain the curvature difference set. This can be achieved in the following way: First, all connection points of adjacent curve segments are identified from the two-dimensional edge curve set. The connection point is the end point of the previous curve segment and the starting point of the next curve segment. Then, for each connection point, the curvature value corresponding to the end point of the previous curve segment and the curvature value corresponding to the starting point of the next curve segment are extracted from the discrete curvature sequence, and the absolute difference between the two curvature values ​​is calculated, i.e., the larger value minus the smaller value. The absolute difference of curvature corresponding to all connection points is recorded in the natural order of the connection points on the seal profile, and each difference is associated with the coordinate position of the connection point, thereby forming the curvature difference set.

[0028] In specific implementation, the curvature difference parameters of the local bending state of the seal can be generated by summarizing all curvature difference values ​​in the curvature difference value set according to the corresponding connection point positions and difference amplitudes. This can be achieved by: traversing each record in the curvature difference value set, dividing the absolute curvature difference value in the record into multiple amplitude levels according to its numerical value, for example, zero to threshold one is a slight fluctuation level, threshold one to threshold two is a medium fluctuation level, and exceeding threshold two is a severe fluctuation level. At the same time, the connection point position coordinates in the record are mapped to the actual geometric space position of the seal. The preset method for threshold one and threshold two is, for example, taking the same batch... The maximum curvature difference of the normal sample of the seal is used as threshold one, and the average of the maximum value of the normal sample and the minimum value of the defective sample is used as threshold two. If there is no defective sample, the maximum allowable curvature change value is calculated based on the elastic limit of the seal material. 60% of this value is set as threshold one and 80% as threshold two, and these values ​​are preset and fixed for use. Then, the amplitude level and geometric position of all connection points are jointly statistically analyzed to form a summary data structure that includes the distribution of fluctuation position and fluctuation intensity. This summary data structure serves as the curvature fluctuation parameter of the local bending state of the seal. Other methods can also be used in other embodiments, which are not limited here.

[0029] It should be noted that, in this application, the discrete curvature sequence refers to a numerical sequence characterizing the degree of curvature at each pixel point on each curve segment in the set of two-dimensional edge curves; the connection point refers to the endpoint position where two adjacent curve segments in the set of two-dimensional edge curves connect to each other; the curvature difference set refers to a summary record of multiple absolute differences characterizing the degree of curvature change between two adjacent curve segments at each connection point; the connection point position refers to the geometric coordinate information of the connection point on the contour of the seal; the difference amplitude refers to the fluctuation intensity level to which the absolute curvature difference at each connection point belongs; and the curvature fluctuation parameter refers to a summary data structure characterizing the severity of curvature change and its spatial distribution law at local bends on the entire outer contour of the seal.

[0030] In some embodiments, generating the visual deformation field of the corresponding seal based on the curvature fluctuation parameters and the spatial height distribution set can be achieved by the following steps: Based on the fluctuation intensity level in the curvature fluctuation parameters, assign a corresponding deformation energy weight to each connection point location; The deformation energy weights are mapped to the corresponding coordinate regions of the spatial height distribution set according to the connection point positions, and the relative height values ​​within the region are incrementally superimposed. The superimposed spatial height distribution set is used as the visual deformation field of the seal.

[0031] In specific implementation, the deformation energy weight assigned to each connection point position according to the fluctuation intensity level in the curvature fluctuation parameters can be achieved in the following way: traverse each connection point position and its corresponding fluctuation intensity level in the curvature fluctuation parameters. If the fluctuation intensity level is a slight fluctuation level, assign a first weight value to the connection point position; if the fluctuation intensity level is a medium fluctuation level, assign a second weight value; if the fluctuation intensity level is a severe fluctuation level, assign a third weight value. The first weight value is less than the second weight value, the second weight value is less than the third weight value, and the first, second, and third weight values ​​are all floating-point numbers between zero and one, such as 0.2, 0.6, and 1.0, respectively, thereby obtaining the deformation energy weight of each connection point position. Other methods can also be used in other embodiments, which are not limited here.

[0032] In specific implementation, the deformation energy weights are mapped to the corresponding coordinate regions of the spatial height distribution set according to the connection point positions. The incremental superposition of the relative height values ​​within this region can be achieved in the following way: For each connection point position, its geometric coordinate information on the seal contour is converted into pixel coordinates in the spatial height distribution set. A square neighborhood with an odd number of pixels on each side, for example, five pixels, is taken as the center of this pixel coordinate. The deformation energy weights assigned to this connection point position are used as a basic incremental value. The relative height value of each pixel in the square neighborhood is increased by the product of the basic incremental value and the Gaussian decay coefficient. The Gaussian decay coefficient gradually decreases from the center to the edge, with the Gaussian decay coefficient at the center being 1 and the Gaussian decay coefficient at the edge approaching 0, so that the influence of each connection point position is distributed in a way that gradually weakens outward from the connection point. The above operation is performed sequentially on all connection point positions to achieve incremental superposition of the relative height values ​​of multiple local regions in the spatial height distribution set, which will not be elaborated here.

[0033] In specific implementation, the superimposed spatial height distribution set can be used as the visual deformation field of the seal in the following way: after completing the superposition of the height increments of all connection point positions, the current superimposed spatial height distribution set is directly marked as the visual deformation field of the seal. This visual deformation field has the same data structure as the original spatial height distribution set, that is, each pixel coordinate still records a relative height value, but this relative height value has been superimposed with the local bending deformation energy reflected by the curvature fluctuation parameter, so that the visual deformation field can simultaneously characterize the original height distribution of the seal and the additional deformation amplitude caused by local bending. Other methods can also be used in other embodiments, which are not limited here.

[0034] It should be noted that, in this application, the fluctuation intensity level refers to a classification identifier characterizing the degree of curvature change at the connection point; the deformation energy weight refers to an intensity coefficient characterizing the influence of each connection point position on the spatial height distribution set; the relative height value refers to the numerical value of the height of the sealing surface relative to the conveyor line plane at each pixel position in the spatial height distribution set, serving as the basic data unit describing the three-dimensional morphology of the sealing component; the superimposed spatial height distribution set refers to the updated result characterizing the original spatial height distribution set after the deformation energy weight increments at each connection point position, wherein the relative height value of each pixel coordinate has incorporated additional deformation information generated by local bending; and the visual deformation field refers to a complete three-dimensional morphological data structure characterizing the sealing component as a whole, simultaneously containing both the original height distribution and local bending deformation information.

[0035] In step 103, based on the degree of deformation continuity in different regions of the visual deformation field, the stable support region and the elastic rebound region of the seal are divided, and an evaluation model of gripping stability is constructed based on the boundary continuity of the stable support region.

[0036] In some embodiments, the division of the stable support region and the elastic rebound region of the seal according to the degree of deformation continuity in different regions of the visual deformation field can be achieved by the following steps: Calculate the standard deviation of the relative height value within each local window in the visual deformation field, and use it as an index of the deformation continuity of that window; All local windows with a deformation continuity index higher than a preset threshold are merged into the stable support area of ​​the seal, and all local windows with a deformation continuity index lower than a preset threshold are merged into the elastic rebound area of ​​the seal. Output a binary mask for the stable support region and a binary mask for the elastic rebound region.

[0037] In specific implementation, the standard deviation of the relative height values ​​within each local window in the visual deformation field is calculated as an index of the deformation continuity of that window. This can be achieved in the following way: the visual deformation field is regarded as a two-dimensional grid, with each grid point storing a relative height value. A square sliding window with an odd number of pixels on each side, such as seven pixels on each side, is used. Starting from the top left corner of the visual deformation field, each window position is traversed sequentially from left to right and from top to bottom. For each window, the relative height values ​​of all grid points within the window are extracted, and the standard deviation of these relative height values ​​is calculated. The calculated standard deviation is used as the deformation continuity index corresponding to the center pixel of the window. After the traversal is completed, a graph of deformation continuity index with the same size as the visual deformation field is obtained, where the value of each pixel position represents the deformation continuity within the local window at that position.

[0038] In specific implementation, all local windows with deformation continuity indices higher than a preset threshold are merged into a stable support region of the seal, and all local windows with deformation continuity indices lower than a preset threshold are merged into an elastic rebound region of the seal. This can be achieved in the following way: First, a preset threshold is set, which can be obtained through offline experimental calibration. For example, the average standard deviation of all windows in the visual deformation field of the same batch of seals under natural flat conditions without external force is taken as the threshold. Then, each pixel position in the deformation continuity index map is traversed. If the deformation continuity index at that position is greater than the preset threshold, the pixel is marked as a stable candidate point; otherwise, it is marked as an elastic candidate point. Connected domain merging is performed on all stable candidate points using a four-connected or eight-connected region growing algorithm to obtain several connected stable candidate regions. The region with the largest area is taken as the stable support region of the seal. The same connected domain merging method is used on all elastic candidate points to obtain the elastic rebound region of the seal. The elastic rebound region may contain multiple scattered connected sub-regions, which are all merged into the overall elastic rebound region.

[0039] In specific implementation, the binary masks for the stable support region and the elastic rebound region can be implemented as follows: Create a blank binary image with the same size as the visual deformation field as the binary mask for the stable support region, assigning a value of 1 to all pixels belonging to the stable support region and a value of 0 to the remaining pixels; then create another identical blank binary image as the binary mask for the elastic rebound region, assigning a value of 1 to all pixels belonging to the elastic rebound region and a value of 0 to the remaining pixels; output these two binary masks for subsequent steps to construct the evaluation model for grasping stability and for indexing and calculating the corresponding regions when determining the offset compensation amount; other methods can also be used in other embodiments, which are not limited here.

[0040] It should be noted that the deformation continuity index in this application refers to the degree of dispersion of the relative height value distribution within each local window in the visual deformation field, serving as the basis for judging whether the deformation of the window region is continuous; the preset threshold in this application refers to the critical value used to distinguish between high and low deformation continuity; the stable support region in this application refers to the part of the seal suitable for providing stable gripping support; the elastic rebound region in this application refers to the part of the seal where deformation is discontinuous and prone to elastic rebound; the binary mask of the stable support region in this application refers to the binary image used to identify the pixel position distribution of the stable support region in the visual deformation field; the binary mask of the elastic rebound region in this application refers to the binary image used to identify the pixel position distribution of the elastic rebound region in the visual deformation field.

[0041] In some embodiments, reference Figure 3 As shown, this figure is an exemplary flowchart of constructing an evaluation model for grasping stability in some embodiments of this application. The evaluation model for grasping stability based on the boundary continuity of the stable support region can be implemented by the following steps: Extract the sequence of outer boundary contour points from the binary mask of the stable support region; Determine the boundary continuity index of the outer boundary contour point sequence; The boundary continuity index and the area parameter of the stable support region are weighted and fused to construct an evaluation model for outputting the grasping stability.

[0042] In specific implementation, the extraction of the outer boundary contour point sequence from the binary mask of the stable support region can be achieved in the following way: the binary mask of the stable support region is used as input, where the pixel value in the stable support region is 1 and the rest are 0. The Moore neighborhood boundary tracing algorithm is used to scan from the upper left corner of the binary mask, find the first pixel with a value of 1 as the starting boundary point, and then trace the next boundary point in a clockwise direction. That is, each time the eight neighboring pixels of the current boundary point are checked for points with a value of 1 and satisfying the boundary connectivity condition, until the starting boundary point is returned and the tracing direction is consistent. All the traced boundary points are recorded in the tracing order to form the outer boundary contour point sequence. Each point in the sequence contains its pixel coordinates in the binary mask.

[0043] In specific implementation, the boundary continuity index of the outer boundary contour point sequence can be determined in the following way: traverse the three adjacent points in the outer boundary contour point sequence, take three consecutive points each time, calculate the vertical distance from the middle point to the line connecting the two points before and after, sum all the vertical distances and divide by the total number of boundary points to obtain the average vertical distance of the boundary. The smaller this value, the smoother and more continuous the boundary. At the same time, consider the outer boundary contour point sequence as a closed curve, calculate the ratio of the perimeter of the closed curve to the enclosed area to obtain the boundary compactness. The reciprocal of the average vertical distance of the boundary and the boundary compactness are weighted and summed, for example, each weight is 0.5, to obtain a value between 0 and 1. This value is used as the boundary continuity index. The closer the value is to 1, the more continuous, smooth, and without sudden local concavities and convexities of the boundary. This will not be elaborated further here.

[0044] In specific implementation, the boundary continuity index and the area parameter of the stable support region are weighted and fused to construct an evaluation model for output grasping stability. This can be achieved in the following way: The total number of pixels with a value of 1 in the binary mask of the stable support region is counted, and multiplied by the actual physical area corresponding to each pixel to obtain the physical area parameter of the stable support region. The actual physical area can be obtained through camera calibration; for example, each pixel represents 0.1 square millimeters. The calculated physical area parameter is then normalized, using the standard support area of ​​the same batch of seals, for example, provided by the seal design drawings or the average value obtained from multiple sample measurements, as a benchmark. Divide the physical area parameter of the stable support region by the standard support area to obtain the normalized area ratio, and limit its maximum value to no more than 1; weight the boundary continuity index and the normalized area ratio, for example, the boundary continuity index has a weight of 0.6 and the normalized area ratio has a weight of 0.4, and the weighted sum is used to obtain a gripping stability score between 0 and 1; this score is used as the output of the gripping stability evaluation model and is used for subsequent steps to match and analyze with the current gripping direction of the mechanical gripper. The higher the score, the more suitable the stable support region is as a gripping contact surface; other methods can also be used in other embodiments, which are not limited here.

[0045] It should be noted that the outer boundary contour point sequence in this application refers to a set of ordered pixel coordinates that characterize the geometric shape of the edge of the stable support region; the boundary continuity index in this application refers to a comprehensive value used to quantitatively evaluate the smoothness and compactness of the boundary of the stable support region; the evaluation model in this application refers to a calculation framework used to fuse the boundary continuity index with the area parameter of the stable support region to output a gripping stability score, which is used for subsequent matching analysis with the gripping direction of the mechanical gripper.

[0046] In step 104, the evaluation model of gripping stability is matched with the current gripping direction of the mechanical gripper, and then the offset compensation amount between the gripping direction and the local deformation direction is determined based on the matching analysis results and the elastic rebound area.

[0047] In some embodiments, the following steps can be used to match and analyze the evaluation model of gripping stability with the current gripping direction of the mechanical gripper, and then determine the offset compensation amount between the gripping direction and the local deformation direction based on the matching analysis results and the elastic rebound region: The main direction of the stable support region is extracted from the evaluation model of gripping stability and used as the recommended gripping direction; Obtain the current gripping direction of the mechanical gripper, calculate its angular deviation from the recommended gripping direction, and obtain the direction matching analysis result; Based on the direction matching analysis results and the main direction of local deformation within the elastic rebound area, the offset compensation amount between the clamping direction and the local deformation direction is determined.

[0048] In specific implementation, the main direction of the stable support region can be extracted from the evaluation model of gripping stability as the recommended clamping direction. This can be achieved in the following way: based on the binary mask of the stable support region, calculate the coordinate covariance matrix of all pixels in the region, perform eigenvalue decomposition on the covariance matrix, and take the direction of the eigenvector corresponding to the largest eigenvalue as the main direction of the stable support region. This main direction represents the longest extension direction of the stable support region on the two-dimensional plane. The angle value of this main direction is output as the recommended clamping direction, which is expressed in radians or degrees in the coordinate system of the conveyor line plane, and is used for subsequent matching with the current clamping direction of the mechanical gripper.

[0049] In specific implementation, the current gripping direction of the mechanical gripper is obtained, and the angular deviation between it and the recommended gripping direction is calculated to obtain the direction matching analysis result. This can be achieved in the following way: the direction vector of the center line of the two gripping fingers of the mechanical gripper is read in real time from the robotic arm controller, and the direction vector is converted into the coordinate system of the conveyor line plane to obtain the angle value of the current gripping direction; the absolute difference between the angle value of the recommended gripping direction and the angle value of the current gripping direction is calculated. If the absolute difference is greater than 180 degrees, the difference is subtracted from 360 degrees to obtain the angle deviation value; this angle deviation value is used as the direction matching analysis result. When the angle deviation value is less than a preset deviation threshold, such as 5 degrees, it indicates that the direction matching is good; otherwise, it indicates that compensation is needed.

[0050] In specific implementation, the offset compensation amount between the clamping direction and the local deformation direction can be determined based on the direction matching analysis results and the main local deformation direction within the elastic rebound region. This can be achieved in the following way: Using the binary mask of the elastic rebound region, calculate the height gradient direction of each pixel within that region, i.e., the direction where the relative height value decreases the fastest. Use Gaussian kernel density estimation to statistically analyze all gradient directions, and take the direction with the highest density as the main local deformation direction within the elastic rebound region. First, determine the angle offset compensation amount by combining the angle deviation value from the direction matching analysis results with the main local deformation direction and the current clamping direction. The angles between the two points are weighted and summed, for example, each weight is 0.5, to obtain the final angle offset compensation amount; then the displacement offset compensation amount is determined by calculating the average gradient magnitude of all pixels in the elastic rebound area and multiplying it by a preset material response coefficient to obtain the displacement offset compensation amount in the direction perpendicular to the current clamping direction. The material response coefficient can be obtained by offline testing the relationship between the elastic rebound displacement and gradient magnitude of the sealing rubber material; the angle offset compensation amount and the displacement offset compensation amount are output as the offset compensation amount between the clamping direction and the local deformation direction; other methods can also be used in other embodiments, which are not limited here.

[0051] It should be noted that the recommended clamping direction in this application refers to the longest extension direction of the stable support area on the two-dimensional plane; the direction matching analysis result in this application refers to the quantitative value of the angular deviation between the recommended clamping direction and the current clamping direction of the mechanical gripper; the main direction of local deformation in this application refers to the dominant direction in which the relative height value decreases the fastest within the elastic rebound area, representing the main trend direction of the rebound deformation of the seal in this area; the offset compensation amount in this application refers to the total amount of angular offset and displacement offset that need to be adjusted relative to the local deformation direction, serving as the parameter basis for generating the end-effector grasping compensation pose of the robotic arm.

[0052] In step 105, the grasping compensation pose of the robotic arm end is generated based on the offset compensation amount, and the robotic arm is controlled to move to the target grasping area.

[0053] In some embodiments, generating the grasping compensation pose of the robotic arm end effector based on the offset compensation amount and controlling the robotic arm to move to the target grasping area can be achieved by the following steps: Obtain the angle offset compensation amount and displacement offset compensation amount from the offset compensation amount, and combine them with the current end-effector pose to generate the grasping compensation pose of the end-effector. Calculate the geometric center of the stable support region based on the binary mask, and use the geometric center as the target grasping region. The grasping compensation pose is converted into robotic arm joint motion commands, driving the robotic arm end effector to move to the target grasping area.

[0054] In specific implementation, obtaining the angular offset compensation and displacement offset compensation from the offset compensation amount, and generating the grasping compensation pose of the robotic arm end effector in combination with the current robotic arm end effector pose, can be achieved in the following way: Read the current robotic arm end effector pose data from the robotic arm controller. This pose data includes the position coordinates in the conveyor plane coordinate system and the rotation angle around the vertical axis; superimpose the angular offset compensation from the offset compensation amount onto the current rotation angle of the robotic arm end effector to obtain the compensated rotation angle; decompose the displacement offset compensation from the offset compensation amount into two displacement components: one along the current rotation angle direction and one perpendicular to that direction, and superimpose them onto the corresponding axes of the current position coordinates of the robotic arm end effector to obtain the compensated position coordinates; combine the compensated position coordinates with the compensated rotation angle to form a new pose, which serves as the grasping compensation pose of the robotic arm end effector.

[0055] In specific implementation, the geometric center of the stable support region is calculated based on the binary mask. This geometric center can be used as the target grasping region in the following way: traverse all pixels with a value of 1 in the binary mask of the stable support region, accumulate the x-coordinate and y-coordinate of all pixels respectively, divide the sum of the x-coordinates by the total number of pixels to obtain the x-coordinate of the geometric center, and divide the sum of the y-coordinates by the total number of pixels to obtain the y-coordinate of the geometric center. This geometric center point is the geometric center of the stable support region. The pixel coordinates of this geometric center in the binary mask are transformed into the coordinate system of the robotic arm's conveyor plane through camera calibration parameters to obtain the actual position coordinates of the target grasping region. The circular area with this coordinate as the center and half the width of the seal as the radius is used as the target grasping region.

[0056] In specific implementation, the grasping compensation pose can be converted into robotic arm joint motion commands to drive the robotic arm end effector to move to the target grasping area. This can be achieved in the following way: the inverse kinematics solver of the robotic arm can be used to convert the grasping compensation pose into angular displacement commands for six joints. The angular displacement commands are sent to the servo driver via a real-time industrial Ethernet bus to drive the robotic arm end effector to move from its current position to the target grasping area. During the movement, steps 1 to 4 of claim 1 are repeated every 50 milliseconds to re-acquire the current visual deformation field and calculate a new offset compensation amount. The new compensation amount is compared with the previous compensation amount. If the angle change exceeds two degrees or the displacement change exceeds 0.5 millimeters, the updated grasping compensation pose of the robotic arm end effector is regenerated and adjusted until the robotic arm end effector reaches the target grasping area and the compensation amount change is less than the above threshold, triggering the gripper closure to complete the stable grasping of the new energy vehicle seal. Other methods can also be used in other embodiments, which are not limited here.

[0057] It should be noted that, in this application, the grasping compensation pose refers to the target position and attitude data characterizing the end effector of the robotic arm after adjustment by angular offset compensation and displacement offset compensation; the target grasping area in this application refers to the circular area defined by the geometric center of the stable support area as the center and half the width of the seal as the radius, which is the grasping position range that the end effector of the robotic arm ultimately needs to reach; the robotic arm joint motion command in this application refers to a set of control signals used to convert the grasping compensation pose into the angular displacement of each joint of the robotic arm, which serves as the execution command to drive the servo driver to move the end effector of the robotic arm to the target grasping area.

[0058] In another aspect, in some embodiments, this application provides a machine vision-based system for grasping seals in new energy vehicles, referencing... Figure 4 The figure is a schematic diagram of a machine vision-based new energy vehicle seal gripping system according to some embodiments of this application. The machine vision-based new energy vehicle seal gripping system 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below: The acquisition module 401 in this application is mainly used to acquire real-time surface images of flexible seals for new energy vehicles in the conveyor line, and to construct a set of two-dimensional edge curves and a set of spatial height distributions of the corresponding seals based on the real-time surface images. Processing module 402, in this application, is mainly used to extract the curvature fluctuation parameters of the local bending state of the seal based on the curvature change relationship of adjacent edge segments in the two-dimensional edge curve set, and generate the visual deformation field of the corresponding seal based on the curvature fluctuation parameters and the spatial height distribution set. The processing module 402 described in this application is also used to divide the stable support region and the elastic rebound region of the seal according to the degree of deformation continuity of different regions in the visual deformation field, and to construct an evaluation model of gripping stability based on the boundary continuity of the stable support region. The processing module 402 described in this application is further used to perform a matching analysis between the evaluation model of the gripping stability and the current gripping direction of the mechanical gripper, and then determine the offset compensation amount between the gripping direction and the local deformation direction based on the matching analysis result and the elastic rebound area. The execution module 403 in this application is mainly used to generate the grasping compensation pose of the end effector of the robotic arm based on the offset compensation amount, and to control the robotic arm to move to the target grasping area.

[0059] The foregoing detailed examples of a machine vision-based method and system for grasping seals in new energy vehicles, as provided in the embodiments of this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0060] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device executes the above-described machine vision-based method for grasping new energy vehicle seals.

[0061] In some embodiments, reference Figure 5 The dashed lines in the figure indicate that the unit or module is optional. This figure is a schematic diagram of the computer device implementing the machine vision-based new energy vehicle seal gripping method of this application. The machine vision-based new energy vehicle seal gripping method in the above embodiments can be achieved through… Figure 5 The computer device 500 shown is used to implement this, and the computer device 500 includes at least one processor 501, a memory 502 and at least one communication unit 505. The computer device 500 may be a terminal device, a server or a chip.

[0062] The processor 501 can be a general-purpose processor or a special-purpose processor. For example, the processor 501 can be a central processing unit (CPU). The CPU can be used to control the computer device 500, execute software programs, and process data from the software programs. The computer device 500 may also include a communication unit 505 for inputting (receiving) and outputting (transmitting) signals.

[0063] For example, computer device 500 may be a chip, communication unit 505 may be the input and / or output circuit of the chip, or communication unit 505 may be the communication interface of the chip, and the chip may be a component of terminal device, network device or other device.

[0064] For example, computer device 500 may be a terminal device or a server, and communication unit 505 may be a transceiver of the terminal device or the server, or communication unit 505 may be a transceiver circuit of the terminal device or the server.

[0065] The computer device 500 may include one or more memories 502 storing a program 504. The program 504 can be executed by a processor 501 to generate instructions 503, causing the processor 501 to perform the methods described in the above method embodiments according to the instructions 503. Optionally, the memory 502 may also store data (such as a target audit model). Optionally, the processor 501 may also read data stored in the memory 502, which may be stored at the same storage address as the program 504, or the data may be stored at a different storage address than the program 504.

[0066] The processor 501 and memory 502 can be configured separately or integrated together, for example, integrated on the system-on-chip (SOC) of the terminal device.

[0067] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 501. The processor 501 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0068] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the above-described machine vision-based method for grasping seals of new energy vehicles.

[0070] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0071] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A machine vision-based method for grasping sealing components of new energy vehicles, characterized in that, Includes the following steps: Real-time surface images of flexible seals for new energy vehicles in the conveyor line are obtained, and a set of two-dimensional edge curves and a set of spatial height distributions of the corresponding seals are constructed based on the real-time surface images. Based on the curvature variation relationship between adjacent edge segments in the set of two-dimensional edge curves, the curvature fluctuation parameters of the local bending state of the seal are extracted, and the visual deformation field of the corresponding seal is generated based on the curvature fluctuation parameters and the set of spatial height distribution. Based on the degree of deformation continuity in different regions of the visual deformation field, the stable support region and the elastic rebound region of the seal are divided, and an evaluation model of gripping stability is constructed based on the boundary continuity of the stable support region. The evaluation model for gripping stability is matched and analyzed with the current gripping direction of the mechanical gripper. Then, based on the matching analysis results and the elastic rebound area, the offset compensation amount between the gripping direction and the local deformation direction is determined. Based on the offset compensation amount, the robotic arm end effector is generated to obtain the grasping compensation pose, and the robotic arm is controlled to move to the target grasping area.

2. The method as described in claim 1, characterized in that, Based on the curvature variation relationship between adjacent edge segments in the aforementioned two-dimensional edge curve set, the curvature fluctuation parameters of the local bending state of the seal are extracted, specifically including: Obtain the discrete curvature sequence of each curve segment in the set of two-dimensional edge curves; Traverse all connection points of adjacent curve segments, and calculate the absolute difference of curvature of each connection point on the two adjacent curve segments based on the discrete curvature sequence to obtain a set of curvature difference values. All curvature differences in the curvature difference set are summarized according to the corresponding connection point position and difference amplitude to generate curvature fluctuation parameters of the local bending state of the seal.

3. The method as described in claim 1, characterized in that, Generating the visual deformation field of the corresponding seal based on the curvature fluctuation parameters and the spatial height distribution set specifically includes: Based on the fluctuation intensity level in the curvature fluctuation parameters, assign a corresponding deformation energy weight to each connection point location; The deformation energy weights are mapped to the corresponding coordinate regions of the spatial height distribution set according to the connection point positions, and the relative height values ​​within the region are incrementally superimposed. The superimposed spatial height distribution set is used as the visual deformation field of the seal.

4. The method as described in claim 1, characterized in that, Based on the degree of deformation continuity in different regions of the visual deformation field, the stable support region and elastic rebound region of the seal are specifically divided as follows: Calculate the standard deviation of the relative height value within each local window in the visual deformation field, and use it as an index of the deformation continuity of that window; All local windows with a deformation continuity index higher than a preset threshold are merged into the stable support area of ​​the seal, and all local windows with a deformation continuity index lower than a preset threshold are merged into the elastic rebound area of ​​the seal. Output a binary mask for the stable support region and a binary mask for the elastic rebound region.

5. The method as described in claim 1, characterized in that, The evaluation model for gripping stability, constructed based on the boundary continuity of the stable support region, specifically includes: Extract the sequence of outer boundary contour points from the binary mask of the stable support region; Determine the boundary continuity index of the outer boundary contour point sequence; The boundary continuity index and the area parameter of the stable support region are weighted and fused to construct an evaluation model for outputting the grasping stability.

6. The method as described in claim 1, characterized in that, The evaluation model for gripping stability is matched and analyzed with the current gripping direction of the mechanical gripper. Based on the matching analysis results and the elastic rebound region, the offset compensation amount between the gripping direction and the local deformation direction is determined, specifically including: The main direction of the stable support region is extracted from the evaluation model of gripping stability and used as the recommended gripping direction; Obtain the current gripping direction of the mechanical gripper, calculate its angular deviation from the recommended gripping direction, and obtain the direction matching analysis result; Based on the direction matching analysis results and the main direction of local deformation within the elastic rebound area, the offset compensation amount between the clamping direction and the local deformation direction is determined.

7. The method as described in claim 1, characterized in that, Generating the grasping compensation pose of the robotic arm end effector based on the offset compensation amount, and controlling the robotic arm to move to the target grasping area specifically includes: Obtain the angle offset compensation amount and displacement offset compensation amount from the offset compensation amount, and combine them with the current end-effector pose to generate the grasping compensation pose of the end-effector. Calculate the geometric center of the stable support region based on the binary mask, and use the geometric center as the target grasping region. The grasping compensation pose is converted into robotic arm joint motion commands, driving the robotic arm end effector to move to the target grasping area.

8. A machine vision-based system for gripping sealing components of new energy vehicles, characterized in that, include: The acquisition module is used to acquire real-time surface images of flexible seals for new energy vehicles in the conveyor line, and to construct a set of two-dimensional edge curves and a set of spatial height distributions of the corresponding seals based on the real-time surface images. The processing module is used to extract the curvature fluctuation parameters of the local bending state of the seal based on the curvature change relationship of adjacent edge segments in the set of two-dimensional edge curves, and generate the visual deformation field of the corresponding seal based on the curvature fluctuation parameters and the set of spatial height distribution. The processing module is also used to divide the stable support region and the elastic rebound region of the seal according to the degree of deformation continuity of different regions in the visual deformation field, and to construct an evaluation model of gripping stability based on the boundary continuity of the stable support region. The processing module is also used to perform a matching analysis between the evaluation model of the gripping stability and the current gripping direction of the mechanical gripper, and then determine the offset compensation amount between the gripping direction and the local deformation direction based on the matching analysis result and the elastic rebound area. The execution module is used to generate the grasping compensation pose of the robotic arm end based on the offset compensation amount, and control the robotic arm to move to the target grasping area.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that the computer device performs the machine vision-based method for grasping new energy vehicle seals as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions or code that, when executed on a computer, cause the computer to implement the machine vision-based method for grasping new energy vehicle seals as described in any one of claims 1 to 7.

Images