An irregular object pairing method, device, equipment and storage medium

Abstract

The disclosure provides a pairing method and device of irregular objects, equipment and storage medium, at least two images of an object to be paired are collected, boundary recognition and depth recognition are performed on the at least two images, and the at least two images cover all surfaces of the object to be paired; feature information of the object to be paired is obtained according to the recognition results and pairing instructions are generated, the feature information of the object to be paired is called to perform contour feature matching, texture feature matching and size feature matching; and a comprehensive feature similarity is calculated as a pairing result of the irregular object according to contour feature similarities, texture feature similarities and size feature similarities obtained through matching.

Description

Technical Field

[0001] This disclosure relates to the field of object matching, and more particularly to a method, apparatus, device, and storage medium for matching irregular objects. Background Technology

[0002] Currently, there is no method available on the market for matching irregular objects. The common approach is to have experienced staff perform the matching. This method is limited by the experience and intuition of the matcher, and cannot guarantee the accuracy of the matching results. The accuracy rate cannot be maintained at a stable level. Furthermore, it is limited by the operator's location, making it difficult to perform matching remotely by viewing images. This results in low work efficiency, inconsistent matching quality, and the need for manual selection of irregular objects from the same batch, which consumes a lot of time and effort. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for pairing irregular objects, in order to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this disclosure, a method for pairing irregular objects is provided, wherein the method includes:

[0005] At least two images of the object to be paired are acquired, boundary recognition is performed on the at least two images to obtain boundary recognition results, and depth recognition is performed on the at least two images to obtain depth recognition results; wherein, the at least two images at least cover all surfaces of the object to be paired;

[0006] Based on the boundary recognition result and the depth recognition result, the feature information corresponding to the at least two images is obtained as the first image information;

[0007] In response to detecting that the first image information has reached a set quantity, a pairing instruction is generated;

[0008] In response to the pairing instruction, the first first image information and the second first image information are called from the first image information when a set number of first image information are reached;

[0009] Using the image contour features in the first image information as a benchmark, the similarity of the comprehensive contour features of the first image information and the second image information is compared based on the first image information and the second image information.

[0010] If the comprehensive contour feature similarity reaches the matching value, then the first first image information is used as a benchmark, and the comprehensive feature similarity of the first first image information and the second first image information is compared according to the first first image information and the second first image information.

[0011] If the comprehensive contour feature similarity does not reach the matching value, then the third first image information is called from the first image information that has reached a set number. The image contour features in the first first image information are used as a benchmark to compare the comprehensive contour feature similarity of the first first image and the third first image. If the comprehensive contour feature similarity reaches the matching value, the comprehensive feature similarity comparison is performed until the objects corresponding to the images in the first images that have reached a set number are compared pairwise.

[0012] Based on the comprehensive feature similarity comparison results, a set of matching results with the highest similarity for each object to be paired is output.

[0013] In one possible implementation, at least two images of the object to be paired are acquired, boundary recognition is performed on the at least two images to obtain boundary recognition results, and depth recognition is performed on the at least two images to obtain depth recognition results, including:

[0014] Acquire at least two images of the object to be paired, and edit the at least two images to obtain the contour map and texture map corresponding to the at least two images of the object to be paired;

[0015] Boundary recognition is performed on the contour maps corresponding to at least two images of the object to be paired to obtain boundary recognition results, and depth recognition is performed on the texture maps corresponding to at least two images of the object to be paired to obtain depth recognition results.

[0016] In one possible implementation, obtaining feature information corresponding to the at least two images as first image information based on the boundary recognition result and the depth recognition result includes:

[0017] Based on the boundary recognition results and the depth recognition results, the contour data, contour features, texture data, texture features and size data corresponding to at least two images of the object to be paired are obtained.

[0018] In one possible implementation, the method further includes:

[0019] Based on the contour data corresponding to the first first image information and the contour data corresponding to the second first image information, calculate the contour similarity between the first first image information and at least two images of the object to be paired in the second first image information.

[0020] Weighting coefficients are set for the contour similarity of at least two images of the object to be paired, and the comprehensive contour feature similarity between the first image information and the second image information is calculated by weighting.

[0021] In one embodiment, the method further includes:

[0022] Based on the texture data corresponding to the first image information and the texture data corresponding to the second image information, calculate the texture similarity between the first image information and at least two images of the object to be paired in the second image information;

[0023] For the texture similarity of at least two images of the object to be paired, weight coefficients are set respectively, and the comprehensive texture feature similarity between the first image information and the second image information is calculated by weighting.

[0024] Based on the size data corresponding to the first first image information and the size data corresponding to the second first image information, calculate the size similarity between the first first image information and at least two images of the object to be paired in the second first image information;

[0025] For the size similarity of at least two images of the object to be paired, weight coefficients are set respectively, and the comprehensive size feature similarity between the first image information and the second image information is calculated by weighting.

[0026] Weighting coefficients are set for the comprehensive contour feature similarity, comprehensive texture feature similarity, and comprehensive size feature similarity, and the comprehensive feature similarity of the first image information and the second image information is calculated by weighting.

[0027] In one embodiment, the method further includes:

[0028] The first image information, comprehensive contour feature similarity, comprehensive texture feature similarity, comprehensive size feature similarity, and comprehensive feature similarity data of each object to be paired are uploaded to the database for storage.

[0029] According to a second aspect of this disclosure, an irregular object pairing apparatus is provided, wherein the apparatus includes:

[0030] An image processing unit is configured to acquire at least two images of an object to be paired, perform boundary recognition on the at least two images to obtain boundary recognition results, and perform depth recognition on the at least two images to obtain depth recognition results; wherein the at least two images at least cover all surfaces of the object to be paired; based on the boundary recognition results and the depth recognition results, feature information corresponding to the at least two images is obtained as first image information;

[0031] The first processing unit is configured to generate a pairing instruction in response to detecting that the first image information has reached a set quantity.

[0032] A first pairing unit is configured to, in response to the pairing instruction, call a first first image information and a second first image information from the first image information that has reached a set number;

[0033] The first similarity calculation unit is used to compare the comprehensive contour feature similarity of the first image information and the second image information based on the image contour features in the first image information and the second image information, using the first image information and the second image information as a reference.

[0034] The first pairing unit is further configured to, if the comprehensive contour feature similarity reaches a matching value, use the first first image information as a benchmark and compare the comprehensive feature similarity of the first first image information and the second first image information according to the first first image information and the second first image information;

[0035] If the comprehensive contour feature similarity does not reach the matching value, then the third first image information is called from the first image information that has reached a set number. The image contour features in the first first image information are used as a benchmark to compare the comprehensive contour feature similarity of the first first image and the third first image. If the comprehensive contour feature similarity reaches the matching value, the comprehensive feature similarity comparison is performed until the objects corresponding to the images in the first images that have reached a set number are compared pairwise.

[0036] The result output unit is used to output a set of matching results with the highest similarity for each object to be paired, based on the comprehensive feature similarity comparison results.

[0037] In one possible embodiment, the device further includes:

[0038] The second similarity calculation unit is used to calculate the texture similarity between the first image information and at least two images of the object to be paired in the second image information, based on the texture data corresponding to the first image information and the texture data corresponding to the second image information.

[0039] For the texture similarity of at least two images of the object to be paired, weight coefficients are set respectively, and the comprehensive texture feature similarity between the first image information and the second image information is calculated by weighting.

[0040] The third similarity calculation unit is used to calculate the size similarity between the first image information and at least two images of the object to be paired in the second image information, based on the size data corresponding to the first image information and the size data corresponding to the second image information.

[0041] For the size similarity of at least two images of the object to be paired, weight coefficients are set respectively, and the comprehensive size feature similarity between the first image information and the second image information is calculated by weighting.

[0042] The fourth similarity calculation unit is used to set weight coefficients for the comprehensive contour feature similarity, comprehensive texture feature similarity, and comprehensive size feature similarity, and to calculate the comprehensive feature similarity of the first image information and the second image information by weighting them.

[0043] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0044] At least one processor; and

[0045] A memory communicatively connected to the at least one processor; wherein,

[0046] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0047] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods described in this disclosure.

[0048] This disclosure discloses a method, apparatus, device, and storage medium for pairing irregular objects. By acquiring and recognizing images of the surface of irregular objects, feature data of the irregular objects is obtained. This feature data includes all surface texture features, contour features, and size features. Based on this feature data, pairing rules are used to match all irregular objects in the same batch, selecting the optimal match. This achieves automated pairing of irregular objects, improving pairing efficiency and accuracy, and reducing errors from manual operation.

[0049] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0050] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0051] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0052] Figure 1 A schematic diagram illustrating the implementation flow of a pairing method for irregular objects according to an embodiment of this disclosure is shown;

[0053] Figure 2 This illustration shows a schematic diagram of the walnut pairing process, which is a method for pairing irregular objects according to an embodiment of the present disclosure.

[0054] Figure 3 This illustration shows a schematic diagram of the pairing rule implementation process of a pairing method for irregular objects according to an embodiment of the present disclosure;

[0055] Figure 4 A schematic diagram of the composition structure of a pairing device for irregular objects according to an embodiment of the present disclosure is shown;

[0056] Figure 5 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0057] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0058] Figure 1 This illustration shows a schematic diagram of the implementation flow of a pairing method for irregular objects according to an embodiment of this disclosure; as shown... Figure 1 As shown, the implementation process of a pairing method for irregular objects according to an embodiment of this disclosure includes the following steps:

[0059] Step 101: Acquire at least two images of the object to be paired, perform boundary recognition on the at least two images to obtain boundary recognition results, and perform depth recognition on the at least two images to obtain depth recognition results; wherein, the at least two images cover at least all surfaces of the object to be paired.

[0060] In this embodiment, before pairing irregular objects, it is necessary to collect basic data on the irregular objects. This is done by acquiring images of the irregular objects. Since the relative position of the irregular objects to the camera is fixed, image recognition technology, including depth recognition and boundary recognition, can be used to identify the acquired images and obtain basic information about the irregular objects in the acquired images. At least two images of the irregular objects must cover all surfaces of the irregular objects.

[0061] Preferably, the irregular object can be photographed from multiple angles, for example, by capturing images of the front, back, left, right, top, and bottom of the irregular object, so as to achieve coverage of all surface features of the irregular object.

[0062] Step 102: Based on the boundary recognition result and the depth recognition result, obtain the feature information corresponding to the at least two images as the first image information.

[0063] In this embodiment, image boundary recognition technology and image depth recognition technology are used to obtain the basic feature information of the irregular object corresponding to at least two images of the irregular object collected above, including: contour feature information, texture feature information, and size feature information.

[0064] Step 103: In response to detecting that the first image information has reached a set quantity, a pairing instruction is generated.

[0065] In this embodiment, when the acquired first image information reaches a set quantity, that is, after all the target objects in the same batch have completed feature information acquisition, a pairing instruction is generated. The acquired first image information is then uploaded to a database for storage.

[0066] Step 104: In response to the pairing instruction, the first first image information and the second first image information are called from the first image information that has reached a set number.

[0067] In this embodiment, the first image information of the same batch of objects to be paired in the database is called, that is, the feature information of the objects to be paired. The pairing method adopts a separate comparison method. Preferably, the first object to be paired is selected as the first pairing object, the first image information of the first pairing object is called, and the first image information of other objects to be paired is called as matching objects to be compared with the first image information.

[0068] Step 105: Using the image contour features in the first first image information as a benchmark, compare the similarity of the comprehensive contour features of the first first image information and the second first image information based on the first first image information and the second first image information.

[0069] In this embodiment, the contour feature information in the first image information of the first paired object is used as the comparison benchmark, and the contour similarity between the two paired objects is calculated based on the contour feature information in the second first image information and the contour feature information in the first first image information.

[0070] The first image information contains contour data of at least two faces of the objects to be paired. By comparing the contour data of the at least two faces of the two objects to be paired, the contour similarity of the corresponding faces is calculated. Preferably, different weight coefficients are set for the contour similarity of the at least two faces to calculate the comprehensive contour feature similarity.

[0071] Step 106: If the comprehensive contour feature similarity reaches the matching value, then using the first first image information as a benchmark, the comprehensive feature similarity of the first first image information and the second first image information is compared according to the first first image information and the second first image information.

[0072] In this embodiment, the contour similarity of the two paired objects calculated above is compared with the pairing criteria. If the contour feature similarity reaches the matching value, the feature information in the first first image information is used as a benchmark to compare the corresponding feature similarity between the first first image information and the second first image information, and the comprehensive feature similarity is calculated.

[0073] Step 107: If the comprehensive contour feature similarity does not reach the matching value, then the third first image information is called from the first image information that has reached a set number. The image contour features in the first first image information are used as a benchmark to compare the comprehensive contour feature similarity of the first first image and the third first image. If the comprehensive contour feature similarity reaches the matching value, the comprehensive feature similarity comparison is performed until the objects corresponding to the images in the first images that have reached the set number are compared pairwise.

[0074] In this embodiment, if the overall contour feature similarity does not reach the matching value, the group is determined to be unmatchable, the pairing result is discarded, and the contour feature information in the first image information of the first paired object is used as the benchmark. The first image information of the new paired object is retrieved from the database, the overall contour similarity between the first paired object and the new paired object is calculated, and the overall feature similarity is compared when the overall contour feature similarity reaches the matching value, until the pairwise comparison of all paired objects in this batch is completed.

[0075] Step 108: Output a set of matching results with the highest similarity for each object to be paired, based on the comprehensive feature similarity comparison results.

[0076] In this embodiment, all pairwise comparison results of the batch of objects to be paired are obtained through the operations of steps 101 to 108 above. The set of results with the highest overall similarity of each object to be paired is selected from the set of results as the pairing result of the batch of irregular objects.

[0077] Figure 2This illustration shows a schematic diagram of a walnut pairing process according to an embodiment of the present disclosure for pairing irregular objects; as shown below. Figure 2 As shown, the walnut pairing process of a pairing method for irregular objects includes the following steps:

[0078] Step 201: Data collection for irregular objects.

[0079] In this embodiment, before pairing irregular objects, it is necessary to collect the attribute information of the irregular objects. The collected data includes six-sided images corresponding to the image data in the six directions of the irregular object: top, bottom, left, right, front, and back. Preferably, multiple cameras are used to take pictures of the walnut from these six directions to obtain six-sided images corresponding to the image data in the six directions of the walnut: top, bottom, left, right, front, and back.

[0080] Step 202: Viewing and editing data for irregular objects.

[0081] In this embodiment, the six-sided image of the irregular object collected above is uploaded to a web server for processing. Preferably, the web server uses a Node JS server to call a Python Service to edit the six-sided image of the walnut, including generating corresponding contour images, cropped images, and second-order images. Based on the corresponding contour images, cropped images, and second-order images, the contour data, texture data, and size data of the image are calculated. The contour data, texture data, and size data are then returned to the Node JS server. The Node JS server generates basic data corresponding to the walnut based on the correspondence between the contour data, texture data, and size data of each image and the six-sided image of the walnut. The basic data of the walnut includes the contour data, texture data, and size data corresponding to the six sides of the walnut.

[0082] In this embodiment, the basic data of the irregular object is stored in the data, and the six-sided diagrams corresponding to the irregular object, including the outline diagram, the second-order diagram, and the trimming diagram, are stored in the system file.

[0083] In this embodiment, before processing the six-sided image corresponding to the irregular object, the method further includes numbering the image data of the irregular object and screening the quality of the image data of the irregular object to determine whether it meets the requirements. This includes performing quality detection on the image data, screening out images with lighting problems such as focus blur, green screen, and barring in the six-sided image, and re-collecting the images with quality problems according to the image number.

[0084] Step 203: Pairing is performed using a machine vision pairing algorithm.

[0085] In this embodiment, the machine vision pairing algorithm is an algorithm for pairing irregular objects according to pairing rules. The pairing rules are as follows: based on the texture data, contour data, and size data corresponding to the acquired image data, calculate the corresponding contour similarity, size similarity, and texture similarity; calculate a comprehensive similarity based on the contour similarity, texture similarity, and size similarity; and perform irregular object pairing based on the comprehensive similarity.

[0086] Preferably, a machine vision pairing algorithm is applied during the walnut pairing process. This includes calculating the contour similarity, texture similarity, and size similarity between any two walnuts in the same batch based on the collected basic data (contour data, texture data, and size data), calculating the comprehensive similarity between any two walnuts in the same batch based on the contour similarity, texture similarity, and size similarity, and pairing the walnuts in the same batch according to the comprehensive similarity. The calculated contour similarity, texture similarity, size similarity, comprehensive similarity, and pairing results are then uploaded to a database for storage.

[0087] In this embodiment, before executing the above-mentioned machine vision pairing algorithm, the system detects whether there is a machine vision pairing model corresponding to the above-mentioned machine vision pairing algorithm for the walnut variety of the batch. If there is, the basic data of all the walnuts in the batch are directly input into the machine vision pairing model. The machine vision pairing model automatically generates a set of results, including similarity data between pairs of walnuts in the batch.

[0088] In this embodiment, if there is no machine vision pairing model, that is, the variety of walnuts collected is a new variety, the system calculates the similarity between walnuts according to the machine vision pairing algorithm and outputs a set of results.

[0089] Figure 3 This is a schematic diagram of the pairing rules for a pairing method for irregular objects according to an embodiment of this disclosure, as shown below. Figure 3 As shown in the figure, the pairing rules of a pairing method for irregular objects according to an embodiment of this disclosure include the following steps:

[0090] Step 301: Calculate contour similarity based on contour data.

[0091] In this embodiment, the contour similarity between two irregular objects is calculated based on their contour data. Taking walnuts as an example, the contour similarity between the top and bottom of each pair of walnuts in the batch is calculated based on the contour data in the basic data of each walnut in the batch. When calculating the contour similarity between the left and right sides, since it is difficult to distinguish between the left and right sides, an exhaustive method is used to calculate all combinations, including: left and left, left and right, right and right, right and left. The group with the highest contour similarity is selected as the benchmark, and the corresponding other group is fixed. For example, if the contour similarity between the left and left sides is the highest, the corresponding other group is the right and right, and so on. Similarly, when calculating the contour similarity between the front and back sides, since it is difficult to distinguish between the front and back sides, an exhaustive method is used to calculate all combinations, including: front and front, back and back, front and back, back and front. The group with the highest contour similarity is selected as the benchmark, and the corresponding other group is fixed. For example, if the contour similarity between the front and front sides is the highest, the corresponding other group is the back and back, and so on.

[0092] In this embodiment, based on the six contour similarity results calculated above, they are divided into four groups according to square brackets: [top-top], [bottom-bottom], [left-right (left), right-left (right)], and [front-back (front), back-front (back)]. Four weighting coefficients are then set accordingly to calculate the overall contour similarity. The weighting coefficients can be set based on the proportional relationship between feature points on different faces of the walnut variety, and the weighting coefficients can correspond to this proportional relationship.

[0093] In this embodiment, based on the four sets of results in the brackets above, four boundary value coefficients are set simultaneously. When the contour similarity is lower than the corresponding boundary value, this set of results is filtered out. For example, when the texture feature similarity above is lower than the set texture feature similarity boundary value above, it is determined that the similarity difference between the walnut and the walnut to be paired is too large, and the pairing result between the walnut and the walnut to be paired is deleted.

[0094] Step 302: Calculate texture similarity based on texture data.

[0095] In this embodiment, the texture similarity between two irregular objects is calculated based on their texture data. Taking walnuts as an example, the texture similarity between the top and bottom of each pair of walnuts in the batch is calculated based on the texture data in the basic data of each walnut in the batch. When calculating the texture similarity between the left and right sides, since it is difficult to distinguish between the left and right sides, an exhaustive method is used to calculate all combinations, including: left and left, left and right, right and right, right and left. The group with the highest texture similarity is selected as the benchmark, and the corresponding other group is fixed. For example, if the texture similarity between the left and left sides is the highest, the corresponding other group is right and right, and so on. Similarly, when calculating the texture similarity between the front and back sides, since it is difficult to distinguish between the front and back sides, an exhaustive method is used to calculate all combinations, including: front and front, back and back, front and back, back and front. The group with the highest texture similarity is selected as the benchmark, and the corresponding other group is fixed. For example, if the texture similarity between the front and front sides is the highest, the corresponding other group is back and back, and so on.

[0096] In this embodiment, based on the six texture similarity results calculated above, they are divided into four groups according to square brackets: [top-top], [bottom-bottom], [left-right (left), right-left (right)], and [front-back (front), back-front (back)]. Four weighting coefficients are then set accordingly to calculate the overall texture similarity. The weighting coefficients can be set based on the proportional relationship between feature points on different faces of the walnut variety, and the weighting coefficients can correspond to this proportional relationship.

[0097] In this embodiment, based on the four sets of results in the brackets above, four boundary value coefficients are set simultaneously. When the texture similarity of one set is lower than the corresponding boundary value, the pairing result is filtered out. For example, if the texture feature similarity above is lower than the set texture feature similarity boundary value above, it is determined that the similarity difference between the walnut and the walnut to be paired is too large, and the pairing result between the walnut and the walnut to be paired is deleted.

[0098] Step 303: Calculate size similarity based on size data.

[0099] In this embodiment, the size similarity between two irregular objects is calculated based on their size data. Taking walnuts as an example, the size similarity of the side, belly, and height between every two walnuts in the batch is calculated based on the size data in the basic data of each walnut in the batch. Based on the three sets of size similarity results obtained, three weighting coefficients are set accordingly, and the overall size similarity is calculated by weighting. The weighting coefficients can be set according to the proportional relationship between the size feature points collected from the side, belly, and height of the walnut variety, and the weighting coefficients corresponding to the proportional relationship are set.

[0100] In this embodiment, based on the three sets of size similarity results, three boundary value coefficients are set simultaneously. When one set of size similarity is lower than the corresponding boundary value, the pairing result is filtered out. For example, if the size similarity of an edge is lower than the set edge size similarity boundary value coefficient, it is determined that the similarity difference between the walnut and the walnut to be paired is too large, and the pairing result between the walnut and the walnut to be paired is deleted.

[0101] Step 304: Calculation of overall similarity.

[0102] In this embodiment, weight parameters are set for the calculated contour similarity, texture similarity, and size similarity, and the final similarity data is calculated by weighting them. The weighting coefficients can be set according to the proportional relationship between the contour feature points, texture feature points, and size feature points collected for this walnut variety, and the weighting coefficients corresponding to this proportional relationship can be set.

[0103] The similarity index can also be replaced by the difference index.

[0104] Step 204: Verify the pairing results.

[0105] In this embodiment, the results generated by the above pairing method include the comprehensive similarity results between each pair of walnuts in the batch, as well as the contour similarity, texture similarity, and size similarity results. The two walnuts with the highest comprehensive similarity are selected for pairing. Preferably, the matching data of each pair of walnuts can be manually reviewed. By comparing each walnut with the walnuts with the highest matching degree through the original image, texture image, and contour image, a set of walnuts with the highest matching degree is finally confirmed as the final pairing result. Walnuts that have been confirmed as the final pairing result will become invalid in the pairing list of other walnuts.

[0106] Preferably, the weight information of all walnuts can be collected, and the walnuts that have completed the above pairing can be screened again based on the weight information difference to obtain the optimal matching object for each walnut.

[0107] Step 205: Optimize the pairing model.

[0108] In this embodiment, a machine vision pairing model is trained based on all the data used in the pairing process of irregular objects. Taking walnuts as an example, the model is trained based on all the previously collected data, including the original contour data, texture data, size data, contour similarity (including 6 results for each pair of walnuts), texture similarity (including 6 results for each pair of walnuts), size similarity (including 3 results for each pair of walnuts), and all intermediate features that may be generated in the middle, as well as the pairing rules of irregular objects.

[0109] The training method in this embodiment includes, but is not limited to, general machine learning methods. Preferably, a decision tree algorithm or a neural network, such as a CNN, can be used. After initializing the weights, the basic data of the walnuts is propagated forward through a convolutional layer, a downsampling layer, and a fully connected layer to obtain the output value. The error between the output value and the target value is judged. When the error is greater than the expected value, the error is propagated back into the network. The errors of the fully connected layer, the downsampling layer, and the convolutional layer are calculated in sequence. The error of each layer can be understood as the total error of the network and how much the network should bear. When the error is lower than the expected value, the model training ends.

[0110] Step 206: Print the successful pairing list and picking instructions.

[0111] In this embodiment, based on the pairing results, the data of the successfully paired walnuts is printed out. Manually, all the successfully paired walnuts are selected from the warehouse according to the pairing results and the walnut numbers, and the picking is carried out according to the pairing results to complete the pairing of this batch of walnuts.

[0112] Figure 4 A schematic diagram of the components of a pairing device for irregular objects according to an embodiment of this disclosure is shown; as follows: Figure 4 As shown, the components of a pairing device for irregular objects include:

[0113] Image processing unit 401 is used to acquire at least two images of an object to be paired, perform boundary recognition on the at least two images to obtain boundary recognition results, and perform depth recognition on the at least two images to obtain depth recognition results; wherein, the at least two images at least cover all surfaces of the object to be paired; based on the boundary recognition results and the depth recognition results, feature information corresponding to the at least two images is obtained as first image information.

[0114] The first processing unit 402 is configured to generate a pairing instruction in response to detecting that the first image information has reached a set quantity.

[0115] The first pairing unit 403 is used to, in response to the pairing instruction, call the first first image information and the second first image information from the first image information that has reached a set number.

[0116] The first pairing unit is further configured to, if the comprehensive contour feature similarity reaches a matching value, use the first first image information as a benchmark and compare the comprehensive feature similarity of the first first image information and the second first image information according to the first first image information and the second first image information;

[0117] If the overall contour feature similarity does not reach the matching value, then a third set of the first image information is called from the first image information that has reached a set number. The image contour features in the first first image information are used as a benchmark to compare the overall contour feature similarity of the first first image and the third first image. If the overall contour feature similarity reaches the matching value, the overall feature similarity comparison is performed until the objects corresponding to the images in the first image that have reached the set number are compared pairwise.

[0118] The first similarity calculation unit 404 is used to compare the comprehensive contour feature similarity of the first image information and the second image information based on the image contour features in the first image information and the second image information, using the first image information and the second image information as a reference.

[0119] The second similarity calculation unit 405 is used to calculate the texture similarity between the first image information and at least two images of the object to be paired in the second image information, based on the texture data corresponding to the first image information and the texture data corresponding to the second image information.

[0120] Weighting coefficients are set for the texture similarity of at least two images of the object to be paired, and the comprehensive texture feature similarity between the first image information and the second image information is calculated by weighting.

[0121] The third similarity calculation unit 406 is used to calculate the size similarity between the first image information and at least two images of the object to be paired in the second image information, based on the size data corresponding to the first image information and the size data corresponding to the second image information.

[0122] Weighting coefficients are set for the size similarity of at least two images of the object to be paired, and the combined size feature similarity between the first image information and the second image information is calculated by weighting.

[0123] The fourth similarity calculation unit 407 is used to set weight coefficients for the comprehensive contour feature similarity, comprehensive texture feature similarity and comprehensive size feature similarity respectively, and to calculate the comprehensive feature similarity of the first image information and the second image information by weighting.

[0124] The result output unit 408 is used to output a set of matching results with the highest similarity for each object to be paired, based on the comprehensive feature similarity comparison results.

[0125] In an exemplary embodiment, the image processing unit 401, the first processing unit 402, the first pairing unit 403, the first similarity calculation unit 404, the second similarity calculation unit 405, the third similarity calculation unit 406, the fourth similarity calculation unit 407, and the result output unit 408 may be implemented by one or more central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components.

[0126] Regarding the apparatus in the above embodiments, the specific manner in which each module and unit performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0127] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0128] Figure 5 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0129] like Figure 5As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a memory-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0130] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as a method for pairing irregular objects. For example, in some embodiments, a method for pairing irregular objects can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the method for pairing irregular objects described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform a method for pairing irregular objects by any other suitable means (e.g., by means of firmware).

[0132] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0136] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0137] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0138] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0139] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0140] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for pairing irregular objects, characterized in that, The method includes: At least two images of the object to be paired are acquired, boundary recognition is performed on the at least two images to obtain boundary recognition results, and depth recognition is performed on the at least two images to obtain depth recognition results; wherein, the at least two images at least cover all surfaces of the object to be paired; Based on the boundary recognition result and the depth recognition result, the feature information corresponding to the at least two images is obtained as the first image information; In response to detecting that the first image information has reached a set quantity, a pairing instruction is generated; In response to the pairing instruction, the first image information of the first image information of the first pairing object and the first image information of the other pairing objects are called from the first image information that has reached a set number; Using the image contour features in the first image information of the first paired object as a benchmark, the similarity of the comprehensive contour features of the first image information of the first paired object and the first image information of the other objects to be paired is compared based on the first image information of the first paired object and the first image information of the other objects to be paired. If the comprehensive contour feature similarity reaches the matching value, then the first image information of the first paired object is used as the benchmark, and the comprehensive feature similarity between the first image information of the first paired object and the first image information of the other objects to be paired is compared according to the first image information of the first paired object and the first image information of the other objects to be paired. If the overall contour feature similarity does not reach the matching value, then the first image information of a new object to be paired is called from the first image information that has reached a set number. The image contour features in the first image information of the first paired object are used as a benchmark to compare the overall contour feature similarity between the first paired object and the new object to be paired. If the overall contour feature similarity reaches the matching value, the overall feature similarity comparison is performed until the objects corresponding to the images in the first images that have reached a set number are compared pairwise. Based on the comprehensive feature similarity comparison results, a set of matching results with the highest similarity for each object to be paired is output; Specifically, based on the boundary recognition result and the depth recognition result, contour data, contour features, texture data, texture features and size data corresponding to at least two images of the object to be paired are obtained as the first image information.

2. The method according to claim 1, characterized in that, The process of acquiring at least two images of the object to be paired, performing boundary recognition on the at least two images to obtain boundary recognition results, and performing depth recognition on the at least two images to obtain depth recognition results includes: Acquire at least two images of the object to be paired, and edit the at least two images to obtain the contour map and texture map corresponding to the at least two images of the object to be paired; Boundary recognition is performed on the contour maps corresponding to at least two images of the object to be paired to obtain boundary recognition results, and depth recognition is performed on the texture maps corresponding to at least two images of the object to be paired to obtain depth recognition results.

3. The method according to claim 1, characterized in that, The method further includes: Based on the contour data corresponding to the first image information of the first paired object and the contour data corresponding to the first image information of the other objects to be paired, calculate the contour similarity between the first image information of the first paired object and the first image information of the other objects to be paired, at least two images of the objects to be paired. Weighting coefficients are set for the contour similarity of at least two images of the objects to be paired, and the comprehensive contour feature similarity between the first image information of the first paired object and the first image information of the other objects to be paired is calculated by weighting.

4. The method according to claim 1, characterized in that, The method further includes: Based on the texture data corresponding to the first image information of the first paired object and the texture data corresponding to the first image information of the other paired objects, calculate the texture similarity between the first image information of the first paired object and at least two images of the paired objects in the first image information of the other paired objects; Weighting coefficients are set for the texture similarity of at least two images of the objects to be paired, and the comprehensive texture feature similarity between the first image information of the first paired object and the first image information of the other objects to be paired is calculated by weighting. Based on the size data corresponding to the first image information of the first paired object and the size data corresponding to the first image information of the other objects to be paired, calculate the size similarity between the first image information of the first paired object and the first image information of the other objects to be paired, corresponding to at least two images of the objects to be paired. Weighting coefficients are set for the size similarity of at least two images of the objects to be paired, and the comprehensive size feature similarity between the first image information of the first paired object and the first image information of the other objects to be paired is calculated by weighting. Weighting coefficients are set for the comprehensive contour feature similarity, comprehensive texture feature similarity, and comprehensive size feature similarity, and the comprehensive feature similarity of the first image information of the first paired object and the first image information of the other objects to be paired is calculated by weighting.

5. The method according to claim 4, characterized in that, The method further includes: The first image information, comprehensive contour feature similarity, comprehensive texture feature similarity, comprehensive size feature similarity, and comprehensive feature similarity data of each object to be paired are uploaded to the database for storage.

6. A pairing device for irregular objects, characterized in that, The device includes: An image processing unit is configured to acquire at least two images of an object to be paired, perform boundary recognition on the at least two images to obtain boundary recognition results, and perform depth recognition on the at least two images to obtain depth recognition results; wherein the at least two images at least cover all surfaces of the object to be paired; based on the boundary recognition results and the depth recognition results, feature information corresponding to the at least two images is obtained as first image information; The first processing unit is configured to generate a pairing instruction in response to detecting that the first image information has reached a set quantity. The first pairing unit is configured to, in response to the pairing instruction, call the first image information of the first pairing object and the first image information of other objects to be paired from the first image information that has reached a set number; The first similarity calculation unit is used to compare the comprehensive contour feature similarity between the first image information of the first paired object and the first image information of the other paired objects, based on the image contour features in the first image information of the first paired object and the first image information of the other paired objects. The first pairing unit is further configured to, if the comprehensive contour feature similarity reaches the matching value, use the first image information of the first paired object as a benchmark, and compare the comprehensive feature similarity between the first image information of the first paired object and the first image information of the other paired objects based on the first image information of the first paired object and the first image information of the other paired objects; If the overall contour feature similarity does not reach the matching value, then the first image information of a new object to be paired is called from the first image information that has reached a set number. The image contour features in the first image information of the first paired object are used as a benchmark to compare the overall contour feature similarity between the first paired object and the new object to be paired. If the overall contour feature similarity reaches the matching value, the overall feature similarity comparison is performed until the objects corresponding to the images in the first images that have reached a set number are compared pairwise. The result output unit is used to output a set of matching results with the highest similarity for each object to be paired, based on the comprehensive feature similarity comparison results. The image processing unit is further configured to, based on the boundary recognition result and the depth recognition result, obtain contour data, contour features, texture data, texture features and size data corresponding to at least two images of the object to be paired, as the first image information.

7. The apparatus according to claim 6, characterized in that, The device further includes: The second similarity calculation unit is used to calculate the texture similarity between the first image information of the first paired object and the first image information of the other paired objects based on the texture data corresponding to the first image information of the first paired object and the texture data corresponding to the first image information of the other paired objects; Weighting coefficients are set for the texture similarity of at least two images of the objects to be paired, and the comprehensive texture feature similarity between the first image information of the first paired object and the first image information of the other objects to be paired is calculated by weighting. The third similarity calculation unit is used to calculate the size similarity between the first image information of the first paired object and the first image information of the other paired objects based on the size data corresponding to the first image information of the first paired object and the size data corresponding to the first image information of the other paired objects. Weighting coefficients are set for the size similarity of at least two images of the objects to be paired, and the comprehensive size feature similarity between the first image information of the first paired object and the first image information of the other objects to be paired is calculated by weighting. The fourth similarity calculation unit is used to set weight coefficients for the comprehensive contour feature similarity, comprehensive texture feature similarity, and comprehensive size feature similarity, and to calculate the comprehensive feature similarity between the first image information of the first paired object and the first image information of the other objects to be paired.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to perform the method according to any one of claims 1-5.

Images