Product assembling method, device and equipment and storage medium
By extracting and processing the image features of the components to be assembled, determining and filtering interfering pixel points, extracting edge pixel points, and calculating offset angles, the problem of low accuracy of component assembly angles in the prior art is solved, and assembly accuracy is improved.
Patent Information
- Application Number
- CN202410969878.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-12
AI Technical Summary
When determining the offset angle of the component, the angle accuracy of images with interference points or irregular shapes is poor, and the assembly needs cannot be met.
The image features of the image of the component to be assembled are extracted, divided into sub-images, interfering pixel points are determined and filtered, smoothing processing is performed, edge pixel points are extracted, and offset angle is calculated.
Improves assembly accuracy of product components, and obtains clearer edges through filtering interfering pixel points and smoothing processing, and accurately determines the offset angle.
Smart Images

Figure CN118941519A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing, and in particular to a product assembly method, device, equipment and storage medium. Background Art
[0002] In the manufacturing process of electronic products, it is usually necessary to assemble product parts. When assembling product parts, due to the angle deviation of the part discharge position, it is usually necessary to guide the robot arm to accurately assemble the product parts to the corresponding position after visual processing. Therefore, how to determine the precise component offset angle from the product image is a key step in product component assembly.
[0003] However, the current method for determining component offset angles has poor accuracy for images with interference points or irregular shapes, and cannot meet assembly requirements.
[0004] Therefore, how to determine the precise component offset angle and improve the assembly accuracy of product components has become a technical problem that needs to be solved urgently. Summary of the invention
[0005] The present disclosure provides a product assembly method, device, equipment and storage medium to at least solve the above technical problems existing in the prior art.
[0006] According to a first aspect of the present disclosure, a product assembly method is provided, the method comprising:
[0007] Extracting image features of an image of a component to be assembled, and dividing the image of the component to be assembled into a plurality of sub-images based on the image features;
[0008] Determine first interfering pixel points in the image of the component to be assembled based on each of the sub-images, and perform filtering processing on the first interfering pixel points;
[0009] Smoothing the image of the component to be assembled after filtering the first interfering pixel points to obtain a smoothed image of the component to be assembled;
[0010] Extracting edge pixel points of the smoothed image of the component to be assembled;
[0011] An offset angle of the component to be assembled is determined based on the edge pixel points, and the offset angle is used to assemble the component to be assembled in the target product.
[0012] In one possible implementation manner, extracting image features of the image of the component to be assembled, and dividing the image of the component to be assembled into a plurality of sub-images based on the image features, includes:
[0013] Extracting a set of pixel values of each pixel point of the image of the component to be assembled as an image feature of the image of the component to be assembled;
[0014] Clustering is performed on the pixels of the image of the component to be assembled based on the image features to obtain a plurality of target pixel clusters, and an image composed of the pixels in each target pixel cluster is used as a sub-image of the image of the component to be assembled.
[0015] In one possible implementation, clustering the pixels of the image of the component to be assembled based on the image features to obtain a plurality of target pixel clusters includes:
[0016] Determining a preset number of pixel points from the set as initial cluster centroids;
[0017] Calculating the Euclidean distance between each pixel point in the set and each of the initial cluster centroids respectively;
[0018] For each pixel point in the set, the pixel point is divided into a cluster where the initial clustering centroid closest to the pixel point is located, to obtain multiple clusters;
[0019] Calculate the pixel mean of each cluster;
[0020] If the pixel means of each cluster no longer change, or the number of clustering iterations reaches a preset number, the obtained multiple clusters are used as target pixel clusters, otherwise, return to the step of determining a preset number of pixels from the set as the initial cluster centroids.
[0021] In one possible implementation manner, the determining the first interfering pixel point in the image of the component to be assembled based on each of the sub-images includes:
[0022] The sub-image with the least number of pixels among the sub-images is determined as the target sub-image;
[0023] A pixel point of the target sub-image is determined as a first interference pixel point.
[0024] In one possible implementation manner, the smoothing process is performed on the image of the component to be assembled after filtering the first interfering pixel points to obtain the smoothed image of the component to be assembled, including:
[0025] performing corrosion processing on the image of the component to be assembled after filtering the first interfering pixel points, to obtain the image of the component to be assembled after corrosion;
[0026] Gaussian filtering is performed on the corroded image of the component to be assembled to obtain a smoothed image of the component to be assembled.
[0027] In one possible implementation manner, extracting edge pixel points of the smoothed image of the component to be assembled includes:
[0028] Based on the gradient of the smoothed image of the component to be assembled, an upper edge pixel point set and a lower edge pixel point set of the smoothed image of the component to be assembled are extracted.
[0029] In one possible implementation manner, determining the offset angle of the component to be assembled based on the edge pixel points includes:
[0030] Determine an upper edge line offset angle of the image of the component to be assembled according to the upper edge pixel point set, and determine a lower edge line offset angle of the image of the component to be assembled according to the lower edge pixel point set;
[0031] The average of the upper edge line offset angle and the lower edge line offset angle is calculated as the offset angle of the component to be assembled.
[0032] According to a second aspect of the present disclosure, a product assembly device is provided, the device comprising:
[0033] An image segmentation module, used for extracting image features of the image of the component to be assembled, and dividing the image of the component to be assembled into a plurality of sub-images based on the image features;
[0034] An interference pixel determination module, used to determine first interference pixel points in the image of the component to be assembled based on each of the sub-images, and filter the first interference pixel points;
[0035] A smoothing processing module, used for smoothing the image of the component to be assembled after filtering the first interfering pixel points, to obtain a smoothed image of the component to be assembled;
[0036] An edge pixel extraction module is used to extract edge pixel points of the smoothed image of the component to be assembled;
[0037] The offset angle determination module is used to determine the offset angle of the components to be assembled based on the edge pixel points, and the offset angle is used to assemble the components to be assembled in the target product.
[0038] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0039] at least one processor; and
[0040] a memory communicatively connected to the at least one processor; wherein,
[0041] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.
[0042] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method described in the present disclosure.
[0043] The product assembly method, device, equipment and storage medium disclosed in the present invention extract the image features of the image of the component to be assembled and divide the image of the component to be assembled into multiple sub-images based on the image features, determine the first interference pixel points in the image of the component to be assembled based on each sub-image, filter the first interference pixel points, smooth the image of the component to be assembled after filtering the first interference pixel points, obtain the smoothed image of the component to be assembled, extract the edge pixel points of the smoothed image of the component to be assembled, determine the offset angle of the component to be assembled based on the edge pixel points, and the offset angle is used to assemble the component to be assembled in the target product. By filtering the first interference pixel points, extracting accurate edge pixel points, and determining the accurate offset angle based on the edge pixel points, the assembly accuracy of the product components is improved.
[0044] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:
[0046] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.
[0047] Figure 1 A schematic diagram of assembling components of a notebook is shown;
[0048] Figure 2 A schematic diagram of an implementation process of the product assembly method provided by an embodiment of the present disclosure is shown;
[0049] Figure 3 A comparison diagram of interference pixel filtering before and after the interference pixel filtering provided by the embodiment of the present disclosure is shown;
[0050] Figure 4 A set of comparative schematic diagrams of the images of the components to be assembled provided by the embodiment of the present disclosure before and after smoothing is shown;
[0051] Figure 5 A schematic diagram of the structure of a product assembly device provided by an embodiment of the present disclosure is shown;
[0052] Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0053] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0054] The current method for determining component offset angles has poor accuracy for images with interference points or irregular shapes, and cannot meet assembly requirements. Figure 1 A schematic diagram of assembling notebook components is shown. Figure 1 As shown, the assembly device 110 is used to assemble notebook components. A robot can be installed on the assembly device 110. The robot's mechanical arm 101 can grab the notebook components and place the notebook components in corresponding assembly positions to complete the assembly of the notebook components. Figure 1 As shown, during the assembly process of notebook components, since there is an angle deviation in the component discharge position, it is necessary to determine the accurate offset angle, and then guide the robot arm 101 to accurately place the notebook component 102 at the corresponding position 103. Therefore, in order to determine the accurate component offset angle and improve the assembly accuracy of product components, the present disclosure provides a product assembly method, device, equipment and storage medium. The product assembly method provided by the present disclosure can be applied to electronic devices such as mobile phones, computers and tablet computers.
[0055] The technical solution of the embodiment of the present disclosure will be described below in conjunction with the drawings in the embodiment of the present disclosure.
[0056] Figure 2 A schematic diagram of an implementation process of the product assembly method provided by the embodiment of the present disclosure is shown. Figure 2 As shown, the method includes:
[0057] S201 : extracting image features of an image of a component to be assembled, and dividing the image of the component to be assembled into a plurality of sub-images based on the image features.
[0058] In the present disclosure, images of product parts grabbed by the robot arm can be collected during the assembly process of the target product as images of the parts to be assembled. Specifically, image acquisition devices such as cameras and mobile phone cameras can be used to collect images of the parts to be assembled. The target product can be any product that needs to be assembled, for example, the target product can be a mobile phone and a laptop computer.
[0059] In a possible implementation, extracting image features of the image of the component to be assembled and dividing the image of the component to be assembled into a plurality of sub-images based on the image features may include steps A1-A2:
[0060] Step A1: extracting a set of pixel values of each pixel point of the image of the component to be assembled as an image feature of the image of the component to be assembled.
[0061] In the present disclosure, the pixels of the image of the component to be assembled can be serialized to obtain a set of pixel values of each pixel point of the image of the component to be assembled. For example, taking the pixel point in the upper left corner of the image of the component to be assembled as the initial pixel point of the sequence, a sequence set D = {x1, x2, ..., xm} of m pixels is obtained in order from left to right and from top to bottom, where x1, x2, ..., xm are the pixel values of the first pixel point, the first pixel point, and the mth pixel point in the image of the component to be assembled, respectively.
[0062] Step A2: clustering the pixels of the image of the component to be assembled based on the image features to obtain a plurality of target pixel clusters, and using an image composed of pixels in each target pixel cluster as a sub-image of the image of the component to be assembled.
[0063] In the present disclosure, the pixel points of the image of the components to be assembled can be clustered by spectral clustering, or the pixel points of the image of the components to be assembled can be clustered by connecting adjacent pixel points to form connected components, or the pixel points of the image of the components to be assembled can be clustered by K-value clustering.
[0064] In a possible implementation, clustering the pixels of the image of the component to be assembled based on the image features to obtain a plurality of target pixel clusters may include steps B1-B5:
[0065] Step B1, determining a preset number of pixel points from the set as initial cluster centroids.
[0066] In the present disclosure, the preset number can be set according to the actual application scenario, for example, the preset number can be set to 3 or 5. A preset number of pixels can be randomly selected from the set as the initial clustering centroid. For example, if the preset number is set to 3, 3 pixels {u1, u2, u3} can be randomly selected from the sequence set of pixel points D = {x1, x2, ..., xm} as the initial clustering centroid.
[0067] Step B2, respectively calculating the Euclidean distance between each pixel point in the set and each of the initial cluster centroids.
[0068] Taking the sequence set of pixel points D = {x1, x2, ..., xm} as an example, each pixel in the sequence set D = {x1, x2, ..., xm} can be traversed, and the Euclidean distance between each pixel point in the set and each of the initial cluster centroids can be calculated using the following formula:
[0069]
[0070] Among them, xj is the pixel value of the j-th pixel in the D set, j is the index value of the j-th pixel in the D set, ui is the pixel value of the i-th initial cluster centroid, and di is the Euclidean distance between the i-th initial cluster centroid and the corresponding centroid.
[0071] Step B3: for each pixel point in the set, the pixel point is divided into a cluster where the initial clustering centroid closest to the pixel point is located, to obtain multiple clusters.
[0072] Take the sequence set of pixel points D = {x1, x2, ..., xm} for illustration. If the j-th pixel point in the set D satisfies the following conditions, it can be determined that the j-th pixel point in the set D belongs to cluster F:
[0073] Fj=i,if di=Min(d1,d2,…,dm)
[0074] Among them, Fj is the cluster label, j ranges from 1 to k, Min(d1, d2, …, dm) is the minimum Euclidean distance, and d1, d2, …, dm is the Euclidean distance between the j-th pixel point in the D set and the centroid of each initial cluster.
[0075] Step B4, calculate the pixel mean of each cluster.
[0076] For each cluster, the average value of the pixel values of all pixels in the cluster can be calculated as the pixel mean of the cluster.
[0077] Step B5, if the pixel means of each cluster no longer change, or the number of clustering iterations reaches a preset number, the obtained multiple clusters are used as target pixel clusters, otherwise, return to execute the step of determining a preset number of pixel points from the set as the initial cluster centroid.
[0078] In the present disclosure, the preset number of times can be set according to actual application requirements, for example, it can be set to 40 times or 50 times.
[0079] In the present disclosure, it is possible to judge whether the centroid of the cluster changes according to the pixel mean. If the pixel mean remains unchanged, it indicates that the centroid of the cluster no longer changes. Otherwise, it indicates that the centroid of the cluster is still changing. After calculating the pixel mean of each cluster, if the pixel mean of each cluster no longer changes, or the number of clustering iterations reaches a preset number, the clustering iteration is terminated. Otherwise, the step of determining a preset number of pixel points from the set as the initial cluster centroid is returned to execute, and clustering iteration is performed again until the iteration ends, and a preset number of clusters are output.
[0080] S202: Determine first interfering pixel points in the image of the component to be assembled based on each of the sub-images, and perform filtering processing on the first interfering pixel points.
[0081] In a possible implementation manner, determining the first interfering pixel point in the image of the component to be assembled based on each of the sub-images may include steps C1-C2:
[0082] Step C1: determine the sub-image with the least number of pixels among the sub-images as the target sub-image.
[0083] Step C2: determining a pixel point of the target sub-image as a first interference pixel point.
[0084] In the present disclosure, an image composed of pixels in each target pixel cluster can be used as a sub-image of the image of the component to be assembled. For multiple target pixel clusters, since the number of interfering pixels in the image is generally small, the target pixel cluster with the smallest number of pixels can be used as the cluster of interfering pixels, that is, the pixels in the target pixel cluster with the smallest number of pixels can be determined as the first interfering pixels, and then the first interfering pixels of the cluster can be filtered. Specifically, the pixel value of the first interfering pixel of the cluster can be reset to 0. Figure 3 FIG. 4 shows a comparison diagram of interference pixel filtering before and after the interference pixel filtering provided by the embodiment of the present disclosure. Figure 3 As shown, Figure 3 (a) is the image of the component to be assembled before filtering the first interfering pixel. Figure 3 (b) is the image of the component to be assembled after filtering the first interfering pixel. Figure 3 (a) and Figure 3(b) It can be seen that Figure 3 In (b), the edge of the image of the component to be assembled is clearer after the first interfering pixel is filtered out. In other words, after the first interfering pixel is filtered out, the interference effect of the transition pixel on the edge of the image is eliminated, making the edge of the image of the component to be assembled clearer.
[0085] S203, performing smoothing processing on the image of the component to be assembled after filtering the first interfering pixel points, to obtain a smoothed image of the component to be assembled.
[0086] In a possible implementation manner, the smoothing process of the image of the component to be assembled after filtering the first interfering pixel points to obtain the smoothed image of the component to be assembled may include steps D1-D2:
[0087] Step D1, performing corrosion processing on the image of the component to be assembled after filtering the first interfering pixels, to obtain the corroded image of the component to be assembled.
[0088] Performing corrosion processing on the image of the component to be assembled after filtering the first interfering pixel points can enhance edge features in the image of the component to be assembled, making the edge clearer and more prominent.
[0089] Step D2, performing Gaussian filtering on the eroded image of the component to be assembled to obtain a smoothed image of the component to be assembled.
[0090] In the present disclosure, the following Gaussian filtering formula can be used to perform Gaussian filtering on the image of the component to be assembled after corrosion:
[0091]
[0092] Wherein, image_gaus represents the image after Gaussian filtering, (x, y) is the pixel coordinate of the pixel point in the image of the component to be assembled after corrosion, and the δ parameter represents the concavity of the normal distribution of the Gaussian function.
[0093] In the present disclosure, the image processed by Gaussian filtering can be used as the image of the component to be assembled after smoothing, or the image processed by Gaussian filtering can be filtered for interference pixels, and the filtered and pixelated image can be determined as the image of the component to be assembled after smoothing.
[0094] For example, Figure 4 A set of comparative schematic diagrams of the image of the component to be assembled provided by the embodiment of the present disclosure before and after smoothing are shown, such as Figure 4 As shown, Figure 4 (a) is the image of the assembled component before smoothing. Figure 4 (b) The image of the component to be assembled after smoothing. Figure 4(a) The edge area 401 of the square frame column and Figure 4 (b) The edge region 402 of the square frame column is the corresponding edge region. Figure 4 (a) The edge region 401 and Figure 4 As can be seen from the edge area 402 of (b), Figure 4 The edge of the edge region 402 in (b) is clearer. That is, after the image of the component to be assembled is smoothed, the image has no undulating noise, and the edge of the image of the component to be assembled becomes clearer.
[0095] S204, extracting edge pixel points of the smoothed image of the component to be assembled.
[0096] In the present disclosure, the upper edge pixel point set and the lower edge pixel point set of the smoothed image of the component to be assembled can be extracted based on the gradient of the smoothed image of the component to be assembled. Specifically, the upper edge pixel point set L1 = {(x_up1, y_up1), (x_up2, y_up2), ..., (x_upn, y_upn)} and the lower edge pixel point set L2 = {(x_d1, y_d1), (x_d2, y_d2), ..., (x_dn, y_dn)} can be extracted based on the gradient of the smoothed image of the component to be assembled. Among them, (x_up1, y_up1), (x_up2, y_up2), ..., (x_upn, y_upn) respectively represent the coordinates of the first upper edge pixel point to the nth upper edge pixel point extracted. After extracting the upper edge pixel set and the lower edge pixel set, the slope and intercept of the upper edge straight line equation of the smoothed image of the component to be assembled and the slope and intercept of the lower edge straight line equation of the component to be assembled can be obtained by the least square method. Specifically, if the edge straight line equation is expressed as y=ax+b, where a is the slope of the edge straight line and b is the intercept of the edge straight line, the values of a and b can be determined by the following formula:
[0097]
[0098] Among them, (xi,yi) represents the coordinates of the i-th pixel point in the upper edge pixel point set or the coordinates of the i-th pixel point in the lower edge pixel point set, and n is the number of pixels included in the upper edge pixel point set or the lower edge pixel point set.
[0099] S205: determining an offset angle of the components to be assembled based on the edge pixel points, wherein the offset angle is used to assemble the components to be assembled in the target product.
[0100] In a possible implementation manner, determining the offset angle of the to-be-assembled component based on the edge pixel points may include steps E1-E2:
[0101] Step E1, determining an upper edge line offset angle of the image of the component to be assembled according to the upper edge pixel point set, and determining a lower edge line offset angle of the image of the component to be assembled according to the lower edge pixel point set.
[0102] In the present disclosure, the angle corresponding to the inverse tangent function of the slope of the upper edge straight line equation of the image of the component to be assembled after smoothing can be calculated as the upper edge line offset angle, and the angle corresponding to the inverse tangent function of the slope of the lower edge straight line equation of the image of the component to be assembled after smoothing can be calculated as the lower edge line offset angle.
[0103] Step E2, calculating the average of the upper edge line offset angle and the lower edge line offset angle as the offset angle of the component to be assembled.
[0104] After calculating the offset angle of the component to be assembled, the robot arm that installs the product component can be controlled to adjust the assembly angle of the component so that the assembly angle of the component is aligned with the installation position, thereby achieving accurate installation of the product component.
[0105] The method is adopted to extract the image features of the image of the component to be assembled and divide the image of the component to be assembled into multiple sub-images based on the image features, determine the first interference pixel points in the image of the component to be assembled based on each sub-image, filter the first interference pixel points, smooth the image of the component to be assembled after filtering the first interference pixel points, obtain the smoothed image of the component to be assembled, extract the edge pixel points of the smoothed image of the component to be assembled, determine the offset angle of the component to be assembled based on the edge pixel points, and the offset angle is used to assemble the component to be assembled in the target product. By filtering the first interference pixel points, extracting accurate edge pixel points, and determining the accurate offset angle based on the edge pixel points, the assembly accuracy of the product components is improved.
[0106] Based on the same inventive concept, according to the product assembly method provided by the above embodiment of the present disclosure, correspondingly, another embodiment of the present disclosure further provides a product assembly device, whose structural schematic diagram is shown in FIG. Figure 5 As shown, specifically including:
[0107] An image segmentation module 501 is used to extract image features of an image of a component to be assembled, and to segment the image of the component to be assembled into a plurality of sub-images based on the image features;
[0108] An interference pixel determination module 502 is used to determine first interference pixel points in the image of the component to be assembled based on each of the sub-images, and filter the first interference pixel points;
[0109] A smoothing processing module 503 is used to perform smoothing processing on the image of the component to be assembled after filtering the first interfering pixel points, so as to obtain a smoothed image of the component to be assembled;
[0110] An edge pixel extraction module 504 is used to extract edge pixel points of the smoothed image of the component to be assembled;
[0111] The offset angle determination module 505 is used to determine the offset angle of the components to be assembled based on the edge pixel points, and the offset angle is used to assemble the components to be assembled in the target product.
[0112] The device is used to extract the image features of the image of the component to be assembled and divide the image of the component to be assembled into multiple sub-images based on the image features, determine the first interference pixel points in the image of the component to be assembled based on each sub-image, filter the first interference pixel points, smooth the image of the component to be assembled after filtering the first interference pixel points, obtain the smoothed image of the component to be assembled, extract the edge pixel points of the smoothed image of the component to be assembled, determine the offset angle of the component to be assembled based on the edge pixel points, and the offset angle is used to assemble the component to be assembled in the target product. By filtering the first interference pixel points, extracting accurate edge pixel points, and determining the accurate offset angle based on the edge pixel points, the assembly accuracy of the product components is improved.
[0113] In one possible implementation, the image segmentation module 501 is specifically used to extract a set of pixel values of each pixel point of the image of the component to be assembled as an image feature of the image of the component to be assembled; cluster the pixel points of the image of the component to be assembled based on the image feature to obtain multiple target pixel point clusters, and use an image composed of pixel points in each target pixel point cluster as a sub-image of the image of the component to be assembled.
[0114] In one possible implementation, the image segmentation module 501 is specifically used to determine a preset number of pixel points from the set as initial cluster centroids; respectively calculate the Euclidean distance between each pixel point in the set and each of the initial cluster centroids; for each pixel point in the set, divide the pixel point into a cluster where the initial cluster centroid closest to the pixel point is located, to obtain multiple clusters; calculate the pixel mean of each cluster; if the pixel means of each cluster no longer change, or the number of clustering iterations reaches a preset number, use the obtained multiple clusters as target pixel point clusters, otherwise, return to execute the step of determining a preset number of pixel points from the set as initial cluster centroids.
[0115] In one possible implementation, the interference pixel determination module 502 is specifically configured to determine the sub-image with the least pixels among the sub-images as the target sub-image; and determine the pixels of the target sub-image as the first interference pixels.
[0116] In one possible implementation, the smoothing processing module 503 is specifically used to perform corrosion processing on the image of the components to be assembled after filtering the first interference pixel points to obtain the corroded image of the components to be assembled; and perform Gaussian filtering processing on the corroded image of the components to be assembled to obtain the smoothed image of the components to be assembled.
[0117] In one possible implementation, the edge pixel extraction module 504 is specifically configured to extract an upper edge pixel point set and a lower edge pixel point set of the smoothed image of the component to be assembled based on the gradient of the smoothed image of the component to be assembled.
[0118] In one possible implementation, the offset angle determination module 505 is specifically used to determine the upper edge line offset angle of the image of the component to be assembled based on the upper edge pixel point set, and to determine the lower edge line offset angle of the image of the component to be assembled based on the lower edge pixel point set; and to calculate the average of the upper edge line offset angle and the lower edge line offset angle as the offset angle of the component to be assembled.
[0119] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
[0120] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0121] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0122] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0123] The computing unit 601 may be a variety of general and / or special processing development components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a product assembly method. For example, in some embodiments, the product assembly method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the product assembly method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the product assembly method in any other appropriate manner (e.g., by means of firmware).
[0124] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0125] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0126] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0128] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0129] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0131] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0132] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A product assembly method, characterized in that: The method comprises: Extracting image features of an image of a component to be assembled, and dividing the image of the component to be assembled into a plurality of sub-images based on the image features; Determine first interfering pixel points in the image of the component to be assembled based on each of the sub-images, and perform filtering processing on the first interfering pixel points; Smoothing the image of the component to be assembled after filtering the first interfering pixel points to obtain a smoothed image of the component to be assembled; Extracting edge pixel points of the smoothed image of the component to be assembled; An offset angle of the component to be assembled is determined based on the edge pixel points, and the offset angle is used to assemble the component to be assembled in the target product.
2. The method according to claim 1, characterized in that The step of extracting image features of the image of the component to be assembled, and dividing the image of the component to be assembled into a plurality of sub-images based on the image features, comprises: Extracting a set of pixel values of each pixel point of the image of the component to be assembled as an image feature of the image of the component to be assembled; Clustering is performed on the pixels of the image of the component to be assembled based on the image features to obtain a plurality of target pixel clusters, and an image composed of the pixels in each target pixel cluster is used as a sub-image of the image of the component to be assembled.
3. The method according to claim 2, characterized in that The clustering process of the pixel points of the image of the component to be assembled based on the image features to obtain a plurality of target pixel point clusters includes: Determining a preset number of pixel points from the set as initial cluster centroids; Calculating the Euclidean distance between each pixel point in the set and each of the initial cluster centroids respectively; For each pixel point in the set, the pixel point is divided into a cluster where the initial clustering centroid closest to the pixel point is located, to obtain multiple clusters; Calculate the pixel mean of each cluster; If the pixel means of each cluster no longer change, or the number of clustering iterations reaches a preset number, the obtained multiple clusters are used as target pixel clusters, otherwise, return to the step of determining a preset number of pixels from the set as the initial cluster centroids.
4. The method according to claim 1, characterized in that: The determining of the first interfering pixel point in the image of the component to be assembled based on each of the sub-images includes: The sub-image with the least number of pixels among the sub-images is determined as the target sub-image; A pixel point of the target sub-image is determined as a first interference pixel point.
5. The method according to claim 1, characterized in that The step of smoothing the image of the component to be assembled after filtering the first interfering pixel points to obtain the smoothed image of the component to be assembled includes: performing corrosion processing on the image of the component to be assembled after filtering the first interfering pixel points, to obtain the image of the component to be assembled after corrosion; Gaussian filtering is performed on the corroded image of the component to be assembled to obtain a smoothed image of the component to be assembled.
6. The method according to claim 1, characterized in that The step of extracting edge pixel points of the smoothed image of the component to be assembled comprises: Based on the gradient of the smoothed image of the component to be assembled, an upper edge pixel point set and a lower edge pixel point set of the smoothed image of the component to be assembled are extracted.
7. The method according to claim 6, characterized in that The determining the offset angle of the to-be-assembled component based on the edge pixel points includes: Determine an upper edge line offset angle of the image of the component to be assembled according to the upper edge pixel point set, and determine a lower edge line offset angle of the image of the component to be assembled according to the lower edge pixel point set; The average of the upper edge line offset angle and the lower edge line offset angle is calculated as the offset angle of the component to be assembled.
8. A product assembly device, characterized in that: The device comprises: An image segmentation module, used for extracting image features of the image of the component to be assembled, and dividing the image of the component to be assembled into a plurality of sub-images based on the image features; An interference pixel determination module, used to determine first interference pixel points in the image of the component to be assembled based on each of the sub-images, and filter the first interference pixel points; A smoothing processing module, used for smoothing the image of the component to be assembled after filtering the first interfering pixel points, to obtain a smoothed image of the component to be assembled; An edge pixel extraction module is used to extract edge pixel points of the smoothed image of the component to be assembled; The offset angle determination module is used to determine the offset angle of the components to be assembled based on the edge pixel points, and the offset angle is used to assemble the components to be assembled in the target product.
9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.
Citation Information
Patent Citations
Irregular cigarette packet image fusion edge detection algorithm based on improved Sobel operator
CN108665440A
3D printing nozzle tail end real-time tracking and positioning method based on central point judgment
CN109080144A
Method for extracting white lines in green field under complex light condition
CN112258509A
Paper typical spectrum extraction method and device and related equipment
CN115564743A
Positioning device
JP2000047729A
