Image processing method, electronic device, and storage medium

Abstract

The application provides an image processing method, an electronic device and a storage medium. The image processing method comprises the following steps: performing feature extraction on a standard workpiece image, and constructing a data model based on the extracted feature data; adopting an edge feature matching algorithm to match the data model with a to-be-tested image, and determining positioning information of a workpiece in the to-be-tested image; performing contour extraction on the to-be-tested image according to the positioning information of the workpiece, and obtaining a mask image of the workpiece; and optimizing the mask image by using edge information of the to-be-tested image, and obtaining a contour edge point set of the workpiece. The above method can improve the extraction accuracy of the contour boundary of the workpiece.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, electronic device and storage medium. Background Technology

[0002] Workpiece contour boundary extraction refers to identifying and locating the complete shape of a workpiece in an image. It is widely used in quality monitoring of automated production lines, robot-guided grasping, and workpiece size measurement. The accuracy of workpiece contour extraction determines the accuracy of subsequent inspection and analysis, and is the foundation for ensuring production quality and automation levels.

[0003] In related technologies, workpiece contour extraction typically relies on image segmentation or edge detection techniques. However, these methods struggle to reliably and accurately identify target edges when faced with complex environmental interference, resulting in inaccurate extracted workpiece contour boundaries. Summary of the Invention

[0004] In view of the above, it is necessary to provide an image processing method, electronic device and storage medium, which aim to solve the technical problem of how to improve the accuracy of workpiece contour boundary extraction.

[0005] On one hand, this application provides an image processing method, the method comprising: extracting features from a standard workpiece image and constructing a data model based on the extracted feature data; using an edge feature matching algorithm to match the data model with a test image to determine the positioning information of the workpiece in the test image; extracting the contour of the test image according to the positioning information of the workpiece to obtain a mask image of the workpiece; and optimizing the mask image using the edge information of the test image to obtain a set of contour edge points of the workpiece.

[0006] In some embodiments of this application, the step of extracting features from a standard workpiece image and constructing a data model based on the extracted feature data includes: scaling the standard workpiece image to obtain a scaled image; downsampling the scaled image to generate an image set including at least two resolution images; extracting edge feature data from each image in the image set; performing rotation transformation on the edge feature data of each image according to at least two rotation angles to generate transformed edge data corresponding to each rotation angle; and associating and storing each rotation angle with the corresponding transformed edge data to obtain the data model.

[0007] In some embodiments of this application, the step of extracting edge feature data of each image in the image set includes: calculating gradient information of pixels in each image in the image set, the gradient information including gradient magnitude; and taking pixels with gradient magnitude greater than a first preset threshold as edge feature points, the edge feature data including gradient information of at least two edge feature points.

[0008] In some embodiments of this application, the gradient information further includes: gradient direction. Calculating the gradient information of each pixel in the image set includes: using an edge detection operator to calculate a first gradient component in the horizontal direction and a second gradient component in the vertical direction for each pixel; and calculating the gradient magnitude and gradient direction of each pixel based on the first gradient component and the second gradient component.

[0009] In some embodiments of this application, the step of using an edge feature matching algorithm to match the data model with the image under test and determine the positioning information of the workpiece in the image under test includes: acquiring edge feature data of the image under test, wherein the edge feature data of the image under test includes gradient information of at least two edge feature points; performing similarity calculation between the edge feature data of the image under test and the data model to obtain at least two candidate regions of the image under test and pose parameters and matching scores of each candidate region; selecting a first candidate region with the highest matching score and which does not overlap with other candidate regions from the at least two candidate regions; extracting standard edge points of the data model and real edge points of the image under test based on the pose parameters of the first candidate region; updating the spatial distance error between the standard edge point set and the real edge point set and the pose parameters of the first candidate region by iterative calculation until the spatial distance error meets a preset convergence condition and the updated pose parameters are obtained; and determining the updated pose parameters as the positioning information of the workpiece.

[0010] In some embodiments of this application, the step of extracting the contour of the image to be tested based on the positioning information of the workpiece and obtaining the mask image of the workpiece includes: inputting the positioning information and the image to be tested into a preset model, so that the preset model encodes the image to be tested based on the positioning information and outputs the mask image of the workpiece.

[0011] In some embodiments of this application, optimizing the mask image using the edge information of the image under test to obtain the contour edge point set of the workpiece includes: extracting a preliminary contour point set of the workpiece from the mask image, the preliminary contour point set including at least two preliminary contour points; obtaining a true contour image of the image under test, the true contour image including at least two true contour points; mapping each point in the preliminary contour point set to the true contour image; searching for true contour points corresponding to each point in the preliminary contour point set within the true contour image, and using the true contour points corresponding to each point in the preliminary contour point set as replacement contour points to obtain a replacement contour point set; calculating the coordinate offset of each point in the replacement contour point set; and correcting the coordinates of each point in the replacement contour point set according to the coordinate offset to obtain the contour edge point set of the workpiece.

[0012] In some embodiments of this application, the step of searching for the true contour points corresponding to each point in the preliminary contour point set within the true contour map includes: for each preliminary contour point in the preliminary contour point set, determining a preset neighborhood range centered on the corresponding preliminary contour point in the true contour map; within the preset neighborhood range of each preliminary contour point, searching for the true contour points whose gradient magnitude satisfies preset conditions, and determining the true contour points as replacement contour points for the preliminary contour points.

[0013] On the other hand, this application provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the image processing method described above.

[0014] On the other hand, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor in an electronic device, implements the image processing method described above. In the image processing method provided in this application embodiment, a data model is constructed by extracting features from a standard workpiece image. This data model is then matched with the image to be tested to obtain the workpiece's positioning information within the image to be tested, thus improving the accuracy of workpiece positioning. Based on the positioning information, contour extraction is performed on the image to be tested to generate a mask image of the workpiece, ensuring the integrity of the initial contour region. Furthermore, the edge information contained in the image to be tested itself is used to optimize the mask image, improving the accuracy of contour details and obtaining high-precision workpiece contour information. This improves the accuracy of workpiece contour boundary extraction. Attached Figure Description

[0015] Figure 1 This is a flowchart of an image processing method provided in an embodiment of this application.

[0016] Figure 2 This is a scenario application diagram for digital image contour extraction provided in an embodiment of this application.

[0017] Figure 3 This is a detailed flowchart of step S13 provided in an embodiment of this application.

[0018] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0020] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Unless otherwise specified, the following embodiments and features described herein can be combined with each other.

[0021] In existing technologies, workpiece contour extraction techniques are mainly divided into two categories: traditional image processing methods and deep learning-based segmentation models. However, commonly used edge detection techniques rely on manual parameter tuning and are easily affected by background and noise, making it difficult to accurately extract the contours of target objects in images with complex backgrounds. Furthermore, common deep learning instance segmentation models such as Mask R-CNN and YOLO obtain mask contours with low accuracy, which cannot meet the requirements of precision measurement.

[0022] This application provides an image processing method that can meet the requirements of precision measurement and improve the accuracy of workpiece contour boundary extraction.

[0023] This application provides an image processing method that can be applied to one or more electronic devices. An electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0024] Electronic devices can be any electronic product that allows human-computer interaction with a customer, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0025] Electronic devices may also include network devices and / or client devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0026] The networks in which electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0027] like Figure 1 The diagram shown is a flowchart of an image processing method provided in an embodiment of this application. Depending on different needs, the order of the steps in this flowchart can be adjusted according to actual requirements, and some steps can be omitted. The image processing method is applied to electronic devices, such as… Figure 4 The electronic device 10 shown.

[0028] S10: Extract features from standard workpiece images and construct a data model based on the extracted feature data.

[0029] In some embodiments of this application, an electronic device receives an original image containing a target workpiece input by a user through a user interface. The electronic device performs preliminary extraction on the original image to obtain a target workpiece image. The electronic device then performs perspective transformation, automatic maximum contrast stretching, noise reduction, and normalization on the target workpiece image to obtain a standard workpiece image. Specifically, perspective transformation is performed on the target workpiece image to eliminate visual distortion; maximum contrast stretching is performed to enhance image details; and normalization is performed to standardize the size of the target workpiece image.

[0030] In some embodiments of this application, various noise reduction methods can be used to denoise the target workpiece image, including but not limited to: Gaussian blur algorithm, median filtering algorithm, bilateral filtering algorithm, and nonlocal mean denoising algorithm.

[0031] In some embodiments of this application, feature extraction is performed on a standard workpiece image, and a data model is constructed based on the extracted feature data, including steps S101 to S105.

[0032] S101, The standard workpiece image is scaled to obtain a scaled image.

[0033] For example, the electronic device fixes the width of the image to 640 pixels and scales it proportionally according to the aspect ratio of a standard workpiece image to obtain a scaled image. This process aims to reduce the number of pixels required for subsequent calculations, improve processing efficiency, and maintain the shape of the workpiece.

[0034] S102, the scaled image is downsampled to generate an image set including at least two resolution images.

[0035] In some embodiments of this application, the electronic device performs a downsampling operation on the scaled image to construct an image pyramid structure.

[0036] It should be noted that the image pyramid structure is a data structure used for multi-scale image analysis. Electronic devices generate a series of interconnected images with progressively decreasing resolutions by downsampling and smoothing the original image layer by layer, allowing features in the images to be extracted and processed at different scales.

[0037] In some embodiments of this application, the image pyramid structure comprises at least two image layers with different resolutions. For example, an electronic device constructs a two-layer pyramid structure comprising a low-resolution layer and a high-resolution layer. The low-resolution layer, obtained by downsampling the original scaled image, contains the global contour and main structural information of the workpiece, facilitating rapid initial matching. The high-resolution layer maintains or approximates the resolution of the original scaled image, containing local details and fine edge information of the workpiece, used for high-precision pose calibration. By combining the advantages of different layers, this application can significantly improve positioning accuracy while ensuring matching efficiency.

[0038] S103, extract the edge feature data of each image in the image set.

[0039] In some embodiments of this application, the electronic device performs edge feature extraction on each level of the image pyramid structure.

[0040] In some embodiments of this application, the edge feature data includes gradient information of multiple edge feature points. The electronic device calculates the gradient information of each pixel in each level of the image pyramid structure and selects pixels whose gradient information meets a preset threshold as edge feature points, thereby obtaining the edge feature data of each level of the image.

[0041] It should be noted that gradient is an important feature used to describe the rate of change of image brightness, and in image processing, it is used to characterize the intensity and direction of edges. In industrial inspection scenarios, the contour boundaries of workpieces are usually manifested as changes in image brightness; therefore, gradient information is a key feature for identifying and locating workpiece contours.

[0042] In some embodiments of this application, the electronic device calculates the gradient information of each pixel in the image set. The gradient information includes the gradient magnitude. The electronic device uses pixels with gradient magnitudes greater than a first preset threshold as edge feature points. The edge feature data includes the gradient information of at least two edge feature points.

[0043] In some embodiments of this application, the gradient information further includes the gradient direction. Calculating the gradient information of each pixel in the image set includes: using an edge detection operator to calculate the first gradient component in the horizontal direction and the second gradient component in the vertical direction of each pixel; and calculating the gradient magnitude and gradient direction of each pixel based on the first gradient component and the second gradient component.

[0044] In some embodiments of this application, the edge detection operator includes a Sobel operator. The Sobel operator includes a horizontal convolution kernel and a vertical convolution kernel. The horizontal convolution kernel is used to calculate the gradient in the horizontal direction, and the vertical convolution kernel is used to calculate the gradient in the vertical direction. For each pixel in each image in the image set, the electronic device uses the horizontal convolution kernel to calculate its first gradient component in the horizontal direction and the vertical convolution kernel to calculate its second gradient component in the vertical direction. The gradient magnitude and gradient direction of the pixel are then calculated using the first and second gradient components.

[0045] The formula for calculating the gradient magnitude is as follows: Where (i, j) represents the coordinates of the pixel, G x (i, j) represents the first gradient component of pixel (i, j) in the horizontal direction, G y (i, j) represents the second gradient component of pixel (i, j) in the vertical direction.

[0046] Based on the first and second gradient components mentioned above, the gradient direction is calculated using the following formula: in, This represents the gradient direction.

[0047] In some embodiments of this application, after obtaining the gradient magnitude of each pixel, the electronic device determines the pixels with gradient magnitude greater than a first preset threshold as edge feature points. The setting of the first preset threshold takes into account the quality characteristics of the image and the specific application scenario. For example, for images with high contrast and less noise, the first preset threshold is set to 60%-80% of the maximum gradient magnitude, and for images with low contrast or more noise, the first preset threshold is set to 30%-50% of the maximum gradient magnitude.

[0048] For example, when the workpiece is metal, since there may be interference factors such as reflection and scratches on the surface of the workpiece, by reasonably setting a first preset threshold, the real contour edge and the false edge caused by noise can be effectively distinguished.

[0049] In this embodiment, by using gradient calculation and threshold filtering, this application can accurately extract key edge feature points that characterize the workpiece contour from a complex background.

[0050] S104, rotate and transform the edge feature data of each image according to at least two rotation angles to generate transformed edge data corresponding to each rotation angle.

[0051] In some embodiments of this application, in order to address the arbitrary rotation angles that the workpiece may appear in the image to be tested and to improve the accuracy of the subsequent matching process, the electronic device performs rotation transformation on the edge feature data extracted from each level of the image pyramid structure to construct a feature template library covering multiple rotation angles.

[0052] In some embodiments of this application, the electronic device establishes a two-dimensional Cartesian coordinate system with the geometric center of the standard workpiece image as the rotation center. The rigid rotation of the workpiece in the two-dimensional plane is simulated using a rotation transformation matrix, including: For any edge feature point Rotation angle around the center of rotation New coordinates The following transformation matrix is ​​used to calculate: in, This is the translation compensation amount, used to ensure that the rotation center coincides with the geometric center of the image.

[0053] Following the above embodiments, the electronic device simultaneously adjusts the gradient direction of each edge feature point. The edge feature points are adjusted at rotation angles. The new gradient direction The calculation formula is: in, The gradient direction before rotation.

[0054] In some embodiments of this application, to balance matching accuracy and computational complexity, the electronic device employs a fixed-step rotation strategy. For example, the electronic device generates 360 uniformly distributed rotation angles continuously within the range of 0 to 359 degrees, with a rotation step size of 1 degree. For each rotation angle, the electronic device performs the aforementioned coordinate and gradient direction transformation calculation to generate a transformation edge dataset.

[0055] In some embodiments of this application, the electronic device employs a variable step-size rotation strategy based on computational efficiency. For example, a 2-degree step size is used in the 0-90 degree range, and a 5-degree step size is used in other angle ranges. This ensures basic coverage of all angles while reducing the number of feature templates and improving matching efficiency.

[0056] S105, associate and store each rotation angle with the corresponding transformation edge data to obtain the data model.

[0057] In some embodiments of this application, the electronic device associates and integrates the transformed edge data with its corresponding rotation angle and the image pyramid level information to construct a data model.

[0058] In some embodiments of this application, the electronic device employs a hierarchical image pyramid structure to construct the data model. Specifically, for each resolution level in the image pyramid structure, the electronic device establishes a corresponding feature template set. This feature template set consists of feature templates of the workpiece at different rotation angles within that level. Each feature template stores edge feature data of the workpiece at the corresponding level and at different rotation angles. Within each level, the electronic device establishes a storage structure indexed by rotation angle to manage the feature templates.

[0059] In one specific embodiment, the data model adopts a key-value pair data structure. The key is a string composed of a level identifier and a rotation angle, for example, "L1_45" represents the first pyramid level with a 45-degree rotation angle as the feature template; the value is the transformed edge dataset corresponding to the key, recording the coordinates, gradient magnitude, and gradient direction of all edge feature points under this specific level and angle condition.

[0060] In this embodiment, by constructing a data model with multi-scale and multi-angle features, rapid and accurate positioning of the workpiece contour can be achieved in complex scenarios.

[0061] S11, using an edge feature matching algorithm, the data model is matched with the image to be tested to determine the positioning information of the workpiece in the image to be tested.

[0062] In some embodiments of this application, an edge feature matching algorithm is used to match the data model with the image to be tested, and the positioning information of the workpiece in the image to be tested is determined, including steps S111 to S116.

[0063] S111, Obtain the edge feature data of the image to be tested, wherein the edge feature data of the image to be tested includes gradient information of at least two edge feature points.

[0064] In some embodiments of this application, the electronic device uses the same method as in constructing the data model to extract edge feature data of the image under test. The specific implementation process of this embodiment is described in step S103 above; to avoid repetition, it will not be repeated here.

[0065] S112, the edge feature data of the image to be tested is compared with the data model to calculate the similarity, so as to obtain at least two candidate regions of the image to be tested and the pose parameters and matching scores of each candidate region.

[0066] In some embodiments of this application, the electronic device employs a hierarchical image pyramid structure to construct the data model. The data model stores feature template sets according to different resolution levels. Each feature template set consists of feature templates of the workpiece at different rotation angles within that level. Each feature template stores edge feature data of the workpiece at the corresponding level and different rotation angles. Within each level, a storage structure indexed by rotation angle is established with the electronic device to manage the feature templates.

[0067] In some embodiments of this application, the similarity calculation can employ various matching measurement methods, such as the energy function method based on gradient direction consistency, the Hausdorff distance method based on the spatial distribution of feature points, or the similarity measurement method based on normalized cross-correlation, etc.

[0068] In some embodiments of this application, similarity is calculated using an energy function method based on gradient direction consistency. The formula for the energy function is as follows: in, Represents the energy function. Represents the image to be tested. Represents any feature template in the data model. Indicates the image under test The location coordinates being evaluated in the image under test are measured by the energy function. of Location, Feature Template The degree of matching with the image to be tested. The smaller the value, the higher the match. In the feature template The coordinates of an edge feature point. It is a feature template The set of coordinates of all edge feature points in the above text, summation sign. This indicates that all edge feature points in the feature template are traversed and calculated. The points in the image to be tested A square neighborhood centered on the center; It is a feature template At edge feature points The gradient direction at that location; The image to be tested The gradient direction at the edge feature point t.

[0069] In some embodiments of this application, the electronic device calculates the matching score using the aforementioned energy function formula. To achieve comprehensive search and localization of the target workpiece in the image under test, the electronic device constructs a three-dimensional search space. This three-dimensional search space is a mathematical model established to systematically search for all possible pose states of the workpiece in the image under test. It consists of three parameters: coordinates in the image plane, used to determine the center position of the workpiece; rotation angle about an axis perpendicular to the image plane, used to determine the orientation of the workpiece; and a scale factor, used to accommodate possible size scaling of the workpiece in the image. These three parameters constitute the pose parameters of the workpiece in the image under test.

[0070] The electronic device sets the search range and step size of parameters based on the actual size of the image under test, the known size of the standard workpiece in the data model, and prior knowledge. For example, the search range for coordinates is the entire image area, the search range for rotation angle is set to the full angular range from 0° to 360°, and the search range for scale factor can be determined based on the maximum and minimum possible scaling ratios of the workpiece. Within the search range, the electronic device generates discrete parameter points according to preset step sizes, such as a position step size of 5 pixels, an angle step size of 1°, and a scale step size of 0.05, where each parameter point represents a candidate pose of the workpiece.

[0071] The electronic device traverses all pre-generated candidate poses in the search space and calls the aforementioned energy function to calculate the matching score for each candidate pose. This matching score quantitatively reflects the degree of similarity between the image region to be tested and the data model template at that candidate pose.

[0072] In some embodiments of this application, a matching threshold is set, and all candidate poses with matching scores higher than the threshold are determined as valid detection results. The electronic device defines the image space range corresponding to each valid detection result as a candidate region. The candidate region is a rectangular area calculated based on the coordinate parameters in the pose parameters, the scale factor, and the standard workpiece size, while recording the rotation angle θ and the matching score of the region. Finally, a list of candidate regions containing pose information and matching scores is output. S113, select the first candidate region with the highest matching score from the at least two candidate regions and which does not overlap with other candidate regions.

[0073] In some embodiments of this application, due to factors such as image noise and edge feature similarity, there may be multiple candidate regions with similar pose parameters and high matching scores in the surrounding area of ​​the same workpiece. To avoid repeated detection and determine the final positioning result, the electronic device can use a strategy based on non-maximum suppression to screen candidate regions. The working principle of the non-maximum suppression algorithm is to retain the one with the highest confidence among overlapping detection boxes, while suppressing other detection boxes with high overlap around it.

[0074] For example, the electronic device sorts all candidate regions according to their matching scores from high to low, establishing an ordered list of candidate regions. The candidate region with the highest matching score is initially determined as the current optimal detection result and denoted as the first candidate region. Further, the electronic device calculates the spatial overlap between the first candidate region and other candidate regions in the list. A preset overlap threshold is used; when the spatial overlap between a candidate region and the first candidate region exceeds this threshold, it is determined that the two regions have significant overlap, indicating that they likely correspond to the same workpiece instance. In this case, the electronic device retains the first candidate region with the higher matching score and removes the other overlapping candidate region from the list. After completing the above comparison and removal operations, the electronic device selects the highest-scoring candidate region from the remaining candidate regions again and repeats the same overlap calculation and filtering process until all candidate regions have been traversed.

[0075] This embodiment of the application, through the aforementioned iterative screening mechanism, ensures that the obtained first candidate region not only has the highest global matching score, but also that there are no other candidate regions that highly overlap with it in its vicinity. This eliminates duplicate detection, guarantees the uniqueness and accuracy of the output results, and provides reliable localization information for subsequent contour extraction.

[0076] S114, based on the pose parameters of the first candidate region, extract the standard edge points of the data model and the true edge points of the image to be tested.

[0077] Specifically, the electronic device first extracts standard edge points from the data model that match the pose parameters of the first candidate region. The pose parameters, including rotation angles and scale factors, serve as search criteria. The electronic device then selects the best-matching feature template from a pre-built feature template library to obtain a set of standard edge points. These standard edge points represent the precise geometric contour that the target workpiece should present under ideal imaging conditions.

[0078] In some embodiments of this application, the electronic device extracts the real edge points corresponding to the first candidate region in the image to be tested, defines the bounding box region of the first candidate region in the image space based on the pose parameters of the first candidate region, and performs edge extraction within the bounding box region.

[0079] In some embodiments of this application, the electronic device employs the Canny edge detection algorithm within this bounding box region. A dual-threshold mechanism preserves significant edge structures, ultimately obtaining a set of true edge points from the actual acquired images. These true edge points represent the actual geometric information of the workpiece under real imaging conditions.

[0080] S115, the spatial distance error between the standard edge point set and the real edge point set and the pose parameters of the first candidate region are updated by iterative calculation until the spatial distance error meets the preset convergence condition, and the updated pose parameters are obtained.

[0081] In some embodiments of this application, the electronic device converts the pose parameters of the first candidate region into an initial transformation matrix. ,in For rotation matrix, As a translation vector, the electronic device transforms the standard edge point set into the coordinate system of the image under test according to the initial transformation matrix, thus obtaining the transformed standard edge point set. Furthermore, for each transformed standard edge point, the nearest neighbor real edge point is found to construct a point-to-point relationship. The electronic device calculates the unit normal vector of each point in the real edge point set to obtain the normal vector set. Based on the point-to-point relationship between the standard edge points and the real edge points mentioned above, the objective function is constructed as follows: in, Let R represent the objective function, R represent the rotation matrix, t represent the translation vector, and p represent the translation vector. i Represents the standard edge point set; q i Represents the set of real edge points; n i The set of normal vectors representing the set of true edge points.

[0082] In this embodiment, the optimal transformation matrix that satisfies the preset convergence condition is obtained by iteratively optimizing the rotation matrix R and the translation vector t, and the pose parameters are updated based on the optimal transformation matrix.

[0083] in, This represents the Euclidean distance vector between the transformed standard edge points and the true edge point set, expressed as a vector intersecting the normal vector. A dot product operation is performed to project the error onto the normal direction, forming the normal distance error. This error metric based on normal distance is more targeted than the traditional Euclidean distance, converges faster, and is more effective for edge registration. S116, the updated pose parameters are determined as the positioning information of the workpiece.

[0084] In some embodiments of this application, the updated pose parameters include: coordinates in the image plane to determine the center position of the workpiece; rotation angle about an axis perpendicular to the image plane to determine the orientation of the workpiece; and a scale factor to characterize the degree of scaling of the workpiece.

[0085] In this embodiment, the feature information in the data model is matched with the image to be tested, and non-maximum suppression is used to select the first candidate region with the highest score in the image to be tested. The pose parameters of the first candidate region are iteratively optimized to obtain accurate workpiece positioning information and provide accurate coordinate reference for subsequent contour extraction.

[0086] S12, extract the contour of the image to be tested based on the positioning information of the workpiece, and obtain the mask image of the workpiece.

[0087] In some embodiments of this application, electronic devices can use positioning information to extract contours from images under test through various methods, including but not limited to deep learning-based encoder-decoder architectures, instance segmentation methods based on candidate regions, and visual segmentation networks based on attention mechanisms. The appropriate method can be flexibly selected based on the real-time requirements, hardware resource configuration, and accuracy requirements of the specific application scenario. In some embodiments of this application, the electronic device can employ an instance segmentation model to implement the aforementioned contour extraction process. Specifically, the electronic device integrates the coordinates, rotation angles, and scale factors contained in the positioning information to generate a rotated rectangle aligned with the actual pose of the workpiece. This rectangle serves as spatial position cues to guide segmentation. The spatial position cues and the image to be tested are input into the segmentation model. The image encoder of the segmentation model extracts deep visual features of the image, generating a high-dimensional embedding representation. The cues encoder of the segmentation model encodes the spatial position cues into a cues vector compatible with the image features. Through the feature fusion module in the decoder, the image features and the cues vector are fused at multiple levels, ultimately outputting a mask image of the workpiece.

[0088] S13, the mask image is optimized using the edge information of the image to be tested to obtain the contour edge point set of the workpiece.

[0089] In some embodiments of this application, the electronic device can employ various optimization methods to optimize the mask image to obtain a refined set of contour edge points. For example, the electronic device extracts a preliminary contour point set of the workpiece from the mask image and obtains a true edge map based on the image to be tested. By registering and optimizing the preliminary contour point set with the true edge map, the electronic device gradually adjusts the preliminary contour points to positions that match the true edge features, ultimately outputting a contour edge point set with higher accuracy.

[0090] In other embodiments, the workpiece region determined by the mask image is used as the initial contour, and an energy function is constructed by combining the gradient information of the image to be tested. The contour curve is driven to evolve towards the real edge position through iterative calculation. Finally, the converged contour curve is discretely sampled to obtain an accurate set of contour edge points.

[0091] In other embodiments, edge features are extracted at different levels of the image pyramid, the contour information provided by the mask image is fused with edge details at various scales, a smooth and continuous contour is generated by spline curve fitting, and an optimized contour edge point set is obtained by curvature adaptive sampling.

[0092] In other embodiments, the mask image and the multi-channel features of the image to be tested are input into a lightweight convolutional network. Through end-to-end training, the network learns the mapping relationship from coarse segmentation to fine contour, and directly outputs a refined set of contour edge points, thereby improving contour accuracy while ensuring processing efficiency.

[0093] like Figure 2 As shown, Figure 2 This is an application scenario diagram for digital image contour extraction provided in one embodiment of this application. Figure a is a digital image to be tested, and Figure b is a visualized image of the contour after contour extraction. Figure a contains the number "201" and a missing number "3". The image processing task is to extract the contour of the number "201" in Figure a and visualize it. In this application scenario embodiment, the electronic device constructs a data model based on template images of standard numbers "2", "0", and "1", and processes the image to be tested shown in Figure a using the image processing method described in this application. The electronic device identifies and locates the position of the complete number "201" in the image using an edge feature matching algorithm, while excluding the interference of the missing number "3" on the right. Based on the location result, the electronic device extracts the initial mask image of the number "201" using an instance segmentation model, and then optimizes the mask contour by combining it with the edge information of the image to be tested, finally obtaining a contour edge point set with sub-pixel accuracy. The electronic device connects and draws the extracted contour edge point set in the coordinate system of Figure b, generating the contour visualization result shown in Figure b, which clearly and completely shows the contour features of the number "201", and the contour lines are smooth and accurate.

[0094] In the image processing method provided in this application embodiment, a data model is constructed by extracting features from a standard workpiece image. This data model is then matched with the image to be tested to obtain the workpiece's positioning information within the image to be tested, thus improving the accuracy of workpiece positioning. Based on the positioning information, contour extraction is performed on the image to be tested to generate a mask image of the workpiece, ensuring the integrity of the initial contour region. Furthermore, the edge information contained in the image to be tested itself is used to optimize the mask image, improving the accuracy of contour details and obtaining high-precision workpiece contour information. This improves the accuracy of workpiece contour boundary extraction.

[0095] like Figure 3 As shown, Figure 3 This is a detailed flowchart of step S13 provided in the embodiments of this application, which includes the following steps.

[0096] S131, extract the preliminary contour point set of the workpiece from the mask image, the preliminary contour point set including at least two preliminary contour points.

[0097] In some embodiments of this application, the electronic device extracts a preliminary contour point set of the workpiece from a mask image. Specifically, the electronic device uses a boundary tracking algorithm to systematically traverse the boundary pixels of the mask image to obtain a continuous, closed contour path. The electronic device samples the contour path at preset intervals to obtain a preliminary contour point set composed of multiple discrete points, where the position of each point in the preliminary contour point set is represented by pixel-level coordinates.

[0098] It should be noted that since the mask image itself is a binary image with pixels as the basic unit, the position of each pixel is represented by integer coordinates. Therefore, the coordinates of the contour path extracted and the sampled contour points based on this mask image are also represented with pixel-level precision, i.e., the coordinate values ​​are integers. Although these preliminary contour points completely preserve the overall shape features and topological structure of the workpiece, due to the resolution limitations and blurred classification boundaries of the deep learning segmentation model itself, the edge positions of the output mask image may have a deviation of several pixels, which cannot meet the requirements of high-precision industrial measurement for sub-pixel-level positioning accuracy.

[0099] S132, Obtain the true contour map of the image to be tested, wherein the true contour map includes at least two true contour points.

[0100] In some embodiments of this application, the electronic device extracts true contour information from the image under test using an edge detection algorithm. Specifically, the electronic device uses the Sobel edge detection operator to perform convolution calculations on the original image under test, calculating the gradient component of each pixel using its horizontal and vertical convolution kernels, and then obtaining the gradient magnitude of each point. Based on the calculated gradient magnitude information, the electronic device identifies pixels with gradient magnitudes exceeding a preset threshold as valid edge points, and uses all valid edge points to construct the true contour map. This true contour map is directly derived from the original grayscale information of the image under test, and compared to the contour extracted from the mask image, it can more accurately reflect the actual edge position of the workpiece in the image.

[0101] S133, map each point in the preliminary contour point set to the real contour map.

[0102] In some embodiments of this application, the electronic device locates each pixel-level coordinate point in the preliminary contour point set to the image coordinate system where the real contour image is located through coordinate system transformation. The above mapping process maintains the relative positional relationship of each point, so that each preliminary contour point can find a corresponding search area in the real contour image.

[0103] S134, search for the real contour points corresponding to each point in the preliminary contour point set within the real contour map, and use the real contour points corresponding to each point in the preliminary contour point set as replacement contour points to obtain a replacement contour point set.

[0104] In some embodiments of this application, for each pre-mapped contour point, the electronic device searches for the best-matching real contour point within a defined area around it. Through this point-by-point search, the electronic device replaces each point in the pre-mapped contour point set with the corresponding point found in the real contour map, forming a new set of replacement contour points that is closer to the real edge location.

[0105] In some embodiments of this application, searching for the real contour points corresponding to each point in the preliminary contour point set within the real contour map includes steps S1341 to S1342.

[0106] S1341, For each preliminary contour point in the preliminary contour point set, determine a preset neighborhood range centered on the corresponding preliminary contour point in the real contour map. S1342, within the preset neighborhood of each preliminary contour point, search for the real contour point whose gradient magnitude satisfies the preset condition, and determine the real contour point as the replacement contour point of the preliminary contour point.

[0107] In some embodiments of this application, the electronic device employs a local neighborhood search-based strategy to achieve precise localization of real contour points. For each pre-mapped contour point, the electronic device constructs a neighborhood centered on the coordinates of the pre-mapped contour point and sets the neighborhood range, which can be a 5×5 pixel or 7×7 pixel rectangular area centered on the pre-mapped contour point. Within this neighborhood, the electronic device traverses all real contour points and, by comparing the gradient magnitude features of each point, selects the real contour point with the largest gradient magnitude as the best corresponding point of the pre-mapped contour point. This method, which uses the largest gradient magnitude as the selection criterion, ensures that the found edge points have the most significant edge features, thereby improving the geometric consistency of the contour points and providing a high-quality initial position for subsequent sub-pixel-level optimization.

[0108] S135, calculate the coordinate offset of each point in the set of replacement contour points.

[0109] In some embodiments of this application, the electronic device performs subpixel-level coordinate optimization on the set of replacement contour points. Specifically, the electronic device employs a subpixel positioning formula based on gray-level gradient distribution. This formula calculates the precise coordinate offset by analyzing the gray-level variation characteristics within the neighborhood of each replacement contour point. The subpixel positioning formula is as follows: in, Indicates the coordinate offset. Indicates the integer pixel coordinates of the replaced contour points. The grayscale value at the replaced contour point. This represents the grayscale gradient value at the replaced contour point. and These are the zero-order and first-order image moments in the neighborhood of the replaced contour point, respectively.

[0110] in, A coefficient characterizing the degree of edge blurring. The larger the value, the more blurred the edges. The formula is: The coordinate offset is calculated using the sub-pixel positioning formula described above. It can improve the precision of pixel-level coordinates to the sub-pixel level.

[0111] S136, The coordinates of each point in the replacement contour point set are corrected according to the coordinate offset to obtain the contour edge point set of the workpiece.

[0112] In some embodiments of this application, for each point in the set of replacement contour points, the electronic device performs a vector superposition of its original coordinates and the calculated coordinate offset, as expressed by the formula: in, These are the corrected subpixel-level coordinates. By performing this correction operation on all points in the replacement contour point set, the electronic device finally obtains the contour edge point set of the workpiece. Each point in the contour edge point set is represented with subpixel precision. The embodiments of this application not only completely preserve the geometric shape features of the workpiece, but also significantly improve the edge positioning accuracy, which can meet the high-precision requirements for dimensional measurement in industrial inspection.

[0113] like Figure 4 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. The electronic device 10 can be a computer, mobile phone, tablet computer, laptop computer, server, or other similar device. This application embodiment does not impose any restrictions on the specific type of the electronic device 10.

[0114] exist Figure 4 The electronic device 10 may include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104, and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102, and the input / output interface 104 via the bus 105.

[0115] Communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as Universal Serial Bus (USB) and Controller Area Network (CAN). The wireless communication module may provide one or more wireless communication solutions such as Wireless Fidelity (Wi-Fi), Bluetooth (BT), mobile communication networks, frequency modulation (FM), near field communication (NFC), and infrared (IR).

[0116] Memory 102 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 103, and can be used to store executable programs (e.g., machine instructions) of other running programs, as well as user and application data. The RAM may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0117] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 103. Non-volatile memory can include disk storage devices and flash memory.

[0118] The memory 102 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include multiple instructions that, when executed by the processor 103, can implement an image processing method that is executed on the electronic device 10.

[0119] In other embodiments, such as Figure 4 The electronic device 10 shown also includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 10.

[0120] Processor 103 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0121] The processor 103 provides computing and control capabilities. For example, the processor 103 is used to execute computer programs stored in the memory 102 to implement the image processing method described above.

[0122] The input / output interface 104 is used to provide a channel for user input or output. For example, the input / output interface 104 can be used to connect various input / output devices, such as a mouse, keyboard, touch device, display screen, etc., so that users can enter information or visualize information.

[0123] Bus 105 is used at least to provide a channel for communication between communication modules 101, memory 102, processor 103, and input / output interface 104 in electronic device 10.

[0124] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 10. In other embodiments of this application, the electronic device 10 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0125] This application also provides a computer-readable storage medium storing a computer program, which includes program instructions. When the program instructions are executed, the method implemented can refer to the methods in the above embodiments of this application.

[0126] The computer-readable storage medium can be the internal memory of the electronic device described in the above embodiments, such as the hard disk or memory of the electronic device. Alternatively, the computer-readable storage medium can be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., installed on the electronic device.

[0127] In some embodiments, a computer-readable storage medium may include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of the electronic device, etc.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0129] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0131] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0132] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this application may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. An image processing method, characterized in that, The method includes: The method involves extracting features from a standard workpiece image and constructing a data model based on the extracted feature data. This includes: scaling the standard workpiece image to obtain a scaled image; downsampling the scaled image to generate an image set containing at least two resolution images; extracting edge feature data from each image in the image set; performing rotation transformation on the edge feature data of each image according to at least two rotation angles to generate transformed edge data corresponding to each rotation angle; and associating and storing each rotation angle with the corresponding transformed edge data to obtain the data model. An edge feature matching algorithm is used to match the data model with the image to be tested to determine the positioning information of the workpiece in the image to be tested; Based on the positioning information of the workpiece, the contour of the image to be tested is extracted to obtain the mask image of the workpiece; Optimizing the mask image using the edge information of the image under test to obtain the contour edge point set of the workpiece includes: extracting a preliminary contour point set of the workpiece from the mask image, the preliminary contour point set including at least two preliminary contour points; obtaining a true contour image of the image under test, the true contour image including at least two true contour points; mapping each point in the preliminary contour point set to the true contour image; searching for the true contour points corresponding to each point in the preliminary contour point set in the true contour image, and using the true contour points corresponding to each point in the preliminary contour point set as replacement contour points to obtain a replacement contour point set; calculating the coordinate offset of each point in the replacement contour point set; and correcting the coordinates of each point in the replacement contour point set according to the coordinate offset to obtain the contour edge point set of the workpiece.

2. The image processing method as described in claim 1, characterized in that, The step of extracting edge feature data for each image in the image set includes: Calculate the gradient information of each pixel in the image set, wherein the gradient information includes the gradient magnitude; Pixels whose gradient magnitude is greater than a first preset threshold are taken as edge feature points, and the edge feature data includes gradient information of at least two edge feature points.

3. The image processing method as described in claim 2, characterized in that, The gradient information also includes the gradient direction, and calculating the gradient information of each pixel in the image set includes: The first gradient component in the horizontal direction and the second gradient component in the vertical direction of each pixel are calculated using the edge detection operator. Based on the first gradient component and the second gradient component, the gradient magnitude and gradient direction of each pixel are calculated.

4. The image processing method as described in claim 1, characterized in that, The step of using an edge feature matching algorithm to match the data model with the image to be tested and determining the positioning information of the workpiece in the image to be tested includes: Obtain edge feature data of the image under test, wherein the edge feature data of the image under test includes gradient information of at least two edge feature points; The edge feature data of the image under test is compared with the data model to calculate the similarity, thereby obtaining at least two candidate regions of the image under test and the pose parameters and matching score of each candidate region. From the at least two candidate regions, select the first candidate region with the highest matching score that does not overlap with other candidate regions; Based on the pose parameters of the first candidate region, extract the standard edge points of the data model and the true edge points of the image to be tested; The spatial distance error between the standard edge point set and the real edge point set and the pose parameters of the first candidate region are updated by iterative calculation until the spatial distance error meets the preset convergence condition, and the updated pose parameters are obtained. The updated pose parameters are determined as the positioning information of the workpiece.

5. The image processing method as described in claim 1, characterized in that, The step of extracting the contour of the image to be tested based on the positioning information of the workpiece to obtain the mask image of the workpiece includes: The positioning information and the image to be tested are input into a preset model, so that the preset model encodes the image to be tested according to the positioning information and outputs the mask image of the workpiece.

6. The image processing method as described in claim 1, characterized in that, The step of searching for the true contour points corresponding to each point in the preliminary contour point set within the true contour map includes: For each preliminary contour point in the preliminary contour point set, a preset neighborhood range centered on the corresponding preliminary contour point is determined in the real contour map; Within the preset neighborhood of each preliminary contour point, search for the real contour point whose gradient magnitude satisfies the preset condition, and determine the real contour point as the replacement contour point of the preliminary contour point.

7. An electronic device, characterized in that, The electronic device includes: Memory, storing at least one instruction; and The processor executes the at least one instruction to implement the image processing method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which, when executed by a processor in an electronic device, implements the image processing method as described in any one of claims 1 to 6.

Images