A Method, Device, Electronic Device and Medium for Matching High-Noise Sketch Images

Abstract

The present invention relates to a high-noise sketch image matching method, device, electronic device and medium. The method includes: inputting a high-noise sketch to be detected into a trained neural network model to obtain a mask image of the high-noise sketch to be detected; removing the noise in the high-noise sketch to be detected based on an image restoration algorithm, a filtering algorithm and the mask image; performing image grayscale processing and binary processing on both the high-noise sketch to be detected and a standard sketch sequence; respectively obtaining a first cropped image and a second cropped image according to the contours of the high-noise sketch to be detected and the standard sketch sequence; sequentially matching the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped image to obtain a matching result, and obtaining a standard sketch block of the high-noise sketch to be detected according to the matching result. The present invention can accurately remove the high noise in the sketch, thereby realizing the matching of the sketch.

Description

Technical Field

[0001] The present invention relates to the technical field of sketch retrieval, and in particular, to a method, device, electronic device and medium for matching high-noise sketch images. Background Art

[0002] In industrial production, workpiece processing requires blanking according to drawings. Therefore, it is necessary to compare and judge the drawings with rich data information and standard workpiece drawings to determine whether the processed workpiece drawings meet the processing requirements.

[0003] Problems existing in the prior art: 1) Through manual recognition, it is difficult for the naked eye to recognize the tiny details of the drawings, and it is difficult to achieve accurate retrieval; 2) The method of using the Hog+SVM model to implement image classification and then achieve the purpose of retrieving images is only applicable to the retrieval of simple low-noise images. The high-noise information in the images has a large error on the classification results, and the production efficiency is low. Summary of the Invention

[0004] In view of this, it is necessary to provide a method, device, electronic device and medium for matching high-noise sketch images to accurately remove high noise in the sketch and accurately achieve the purpose of sketch matching.

[0005] To achieve the above object, the present invention provides a method for matching high-noise sketch images, including:

[0006] Input the high-noise sketch to be detected into a trained neural network model, and obtain a mask image of the high-noise sketch to be detected through the trained neural network model;

[0007] Remove the noise in the high-noise sketch to be detected based on an image restoration algorithm, a filtering algorithm and the mask image;

[0008] Perform image grayscale and binarization processing on both the high-noise sketch to be detected and the standard sketch sequence, and respectively obtain the processed high-noise sketch and the processed standard sketch sequence;

[0009] Based on a contour extraction algorithm and the processed high-noise sketch and the processed standard sketch sequence, respectively determine the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence;

[0010] Crop the rectangular area where the contour of the high-noise sketch to be detected is located to obtain a first cropped image, and determine the key point descriptor information of the first cropped image based on the SIFT algorithm;

[0011] Crop the rectangular areas where the contours of the standard sketch sequence are located to obtain second cropped images, and determine the key point descriptor information of each image in the second cropped images based on the SIFT algorithm;

[0012] Match the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images in sequence to obtain a matching result, and obtain the standard sketch block of the high-noise sketch to be detected according to the matching result.

[0013] In some possible implementation manners, removing the noise in the high-noise sketch to be detected based on the image inpainting algorithm and the filtering algorithm includes:

[0014] Remove the numbers and texts in the high-noise sketch to be detected based on the image inpainting algorithm;

[0015] Remove the numbers and texts in the high-noise sketch to be detected based on the Gaussian filtering algorithm.

[0016] In some possible implementation manners, respectively determining the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence based on the contour extraction algorithm includes:

[0017] Determine the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence based on the Canny algorithm and the findContours function in OpenCV.

[0018] In some possible implementation manners, cropping the rectangular area where the contour of the high-noise sketch to be detected is located to obtain a first cropped image includes:

[0019] According to the contour of the high-noise sketch to be detected, obtain the coordinates of the upper left vertex of the circumscribed rectangle corresponding to the contour of the high-noise sketch to be detected and the length and width of the circumscribed rectangle;

[0020] Crop the high-noise sketch to be detected within the circumscribed rectangle area through slicing according to the coordinates of the upper left vertex of the circumscribed rectangle, the length and width to obtain a first cropped image.

[0021] In some possible implementation manners, the neural network model is a PP-OCR neural network model.

[0022] In some possible implementation manners, matching the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images in sequence to obtain a matching result, and obtaining the standard sketch block of the high-noise sketch to be detected according to the matching result includes:

[0023] Input the key point descriptor information of the first cropped image and the key point descriptor information of each image in the second cropped image into a FLANN matcher;

[0024] In the key point descriptor information of each image in the second cropped image, find the first image with the highest matching success rate with the key point descriptor information of the first cropped image;

[0025] Use the first image as the standard sketch block of the high-noise sketch to be detected.

[0026] In some possible implementation manners, the step of finding the first image with the highest matching success rate with the key point descriptor information of the first cropped image in the key point descriptor information of each image in the second cropped image includes:

[0027] Respectively obtain the ratio of the minimum Euclidean distance to the second minimum Euclidean distance between the key points of the first cropped image and the key points of each image in the second cropped image;

[0028] Exclude the corresponding images with the ratio greater than the first preset value from the second cropped image;

[0029] Set the minimum number of matching key points between the key points of the first cropped image and the key points of each image in the second cropped image, and exclude the incorrect key point matches in the second cropped image according to the findHomography function in OpenCV to obtain a third cropped image;

[0030] Use the image with the most matching key points between the key points of the third cropped image and the key points of the first cropped image as the first image.

[0031] On the other hand, the present invention also provides a high-noise sketch image matching device, including:

[0032] A mask image acquisition unit, configured to input the high-noise sketch to be detected into a trained neural network model, and obtain a mask image of the high-noise sketch to be detected through the trained neural network model;

[0033] An image denoising unit, configured to remove the noise in the high-noise sketch to be detected based on an image inpainting algorithm, a filtering algorithm, and the mask image;

[0034] An image preprocessing unit, configured to perform image grayscale processing and binarization processing on both the high-noise sketch to be detected and the standard sketch sequence, respectively obtaining a processed high-noise sketch and a processed standard sketch sequence;

[0035] An image contour acquisition unit, configured to determine the contour of the to-be-detected high-noise sketch and the contours of the standard sketch sequence respectively based on a contour extraction algorithm, the processed high-noise sketch, and the processed standard sketch sequence;

[0036] A high-noise sketch key point information acquisition unit, configured to crop a rectangular area where the contour of the to-be-detected high-noise sketch is located to obtain a first cropped image, and determine the key point descriptor information of the first cropped image based on the SIFT algorithm;

[0037] A standard sketch sequence key point information acquisition unit, configured to crop the rectangular areas where the contours of the standard sketch sequence are located respectively to obtain second cropped images, and determine the key point descriptor information of each image in the second cropped images based on the SIFT algorithm;

[0038] A standard sketch block acquisition unit, configured to sequentially match the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images to obtain a matching result, and obtain the standard sketch block of the to-be-detected high-noise sketch according to the matching result.

[0039] On the other hand, the present invention further provides an electronic device, including a memory and a processor, wherein,

[0040] The memory is configured to store a program;

[0041] The processor is coupled to the memory and configured to execute the program stored in the memory to implement the steps in a high-noise sketch image matching method in any one of the above implementation manners.

[0042] On the other hand, the present invention further provides a computer-readable storage medium, configured to store a computer-readable program or instruction, and when the program or instruction is executed by a processor, it can implement the steps in a high-noise sketch image matching method in any one of the above implementation manners.

[0043] The beneficial effects of adopting the above embodiments are as follows: A high-noise sketch image matching method, device, electronic device, and medium provided by the present invention first obtain a mask image of the detected high-noise sketch through a trained complete neural network model, and then remove the noise in the detected high-noise sketch based on an image restoration algorithm, a filtering algorithm, and the mask image. Then, both the detected high-noise sketch and the standard sketch sequence are subjected to image grayscale conversion and binarization processing. Further, based on a contour extraction algorithm, the contours of the detected high-noise sketch and the standard sketch sequence are obtained, and cropping is performed in the rectangular frame area where each contour is located to obtain a first cropped image and a second cropped image respectively. Then, based on the SIFT algorithm, the key point descriptor information of the first cropped image and the key point descriptor information of each image in the second cropped image are extracted, and the key point descriptor information of the first cropped image is matched with the key point descriptor information of each image in the second cropped image to obtain a matching result, and a standard sketch block of the detected high-noise sketch is obtained according to the matching result. The present invention utilizes text detection technology and image restoration technology to achieve precise removal of high noise in sketches, and further improves the accuracy of sketch retrieval through local feature extraction and matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 FIG. is a flowchart of a method according to an embodiment of a high-noise sketch image matching method provided by the present invention;

[0045] Figure 2 FIG. is a schematic structural diagram of an embodiment of a high-noise sketch image matching device provided by the present invention;

[0046] Figure 3 FIG. is a schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] The following will specifically describe the preferred embodiments of the present invention with reference to the accompanying drawings. The accompanying drawings form a part of this application and are used together with the embodiments of the present invention to explain the principles of the present invention, but are not used to limit the scope of the present invention.

[0048] Figure 1 FIG. is a schematic flowchart of an embodiment of a high-noise sketch image matching method provided by the present invention. As Figure 1 shown, a high-noise sketch image matching method is disclosed, which is characterized by including:

[0049] S101. Input the detected high-noise sketch into a trained complete neural network model, and obtain a mask image of the detected high-noise sketch through the trained complete neural network model;

[0050] S102. Remove the noise in the to-be-detected high-noise sketch based on the image inpainting algorithm, the filtering algorithm, and the mask image;

[0051] S103. Perform image grayscaling and binarization processing on both the to-be-detected high-noise sketch and the standard sketch sequence to obtain the processed high-noise sketch and the processed standard sketch sequence respectively;

[0052] S104. Determine the contour of the to-be-detected high-noise sketch and the contours of the standard sketch sequence respectively based on the contour extraction algorithm, the processed high-noise sketch, and the processed standard sketch sequence;

[0053] S105. Crop the rectangular area where the contour of the to-be-detected high-noise sketch is located to obtain a first cropped image, and determine the key point descriptor information of the first cropped image based on the SIFT algorithm;

[0054] S106. Crop the rectangular areas where the contours of the standard sketch sequence are located respectively to obtain second cropped images, and determine the key point descriptor information of each image in the second cropped images based on the SIFT algorithm;

[0055] S107. Match the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images in sequence to obtain a matching result, and obtain the standard sketch block of the to-be-detected high-noise sketch according to the matching result.

[0056] Compared with the prior art, a high-noise sketch image matching method, device, electronic device, and medium provided in this embodiment first obtain the mask image of the to-be-detected high-noise sketch through a trained complete neural network model, then remove the noise in the to-be-detected high-noise sketch based on the image inpainting algorithm, the filtering algorithm, and the mask image, and then perform image grayscaling and binarization processing on both the to-be-detected high-noise sketch and the standard sketch sequence. Further, based on the contour extraction algorithm, obtain the contours of the to-be-detected high-noise sketch and the standard sketch sequence, and perform cropping in the rectangular frame areas where the respective contours are located to obtain a first cropped image and second cropped images respectively; then extract the key point descriptor information of the first cropped image and the key point descriptor information of each image in the second cropped images based on the SIFT algorithm, match the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images to obtain a matching result, and obtain the standard sketch block of the to-be-detected high-noise sketch according to the matching result. The present invention utilizes text detection technology and image inpainting technology to achieve precise removal of high noise in sketches, and further improves the accuracy of sketch retrieval through local feature extraction and matching.

[0057] It should be noted that the SIFT (Scale-invariant feature transform) algorithm, namely scale-invariant feature transform, is a description used in the field of image processing. This description has scale invariance, can detect key points in an image, and is a local feature descriptor.

[0058] In a specific embodiment of the present invention, in step S101, the high-noise sketch to be detected is input into the trained PP-OCR neural network model. Through this model, the four vertex coordinates of the rectangle frame where the numbers and texts are located in the high-noise sketch to be detected are obtained, and a mask image of the same size with non-zero pixels in the rectangle frame area and zero pixels in the remaining parts is obtained.

[0059] In some embodiments of the present invention, removing the noise in the high-noise sketch to be detected based on the image inpainting algorithm and the filtering algorithm includes:

[0060] Removing the numbers and texts in the high-noise sketch to be detected based on the image inpainting algorithm;

[0061] Removing the numbers and texts in the high-noise sketch to be detected based on the Gaussian filtering algorithm.

[0062] In a specific embodiment of the present invention, in step S102, the mask image in step S101 and the inpaint function in OpenCV are used to remove high-noise such as numbers and texts in the high-noise sketch to be detected, and then Gaussian filtering is used to remove the remaining part of the noise points, obtaining the high-noise sketch to be detected with high-noise such as texts and numbers removed.

[0063] In some embodiments of the present invention, respectively determining the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence based on the contour extraction algorithm includes:

[0064] Determining the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence based on the Canny algorithm and the findContours function in OpenCV.

[0065] In a specific embodiment of the present invention, obtaining the contour of the high-noise sketch to be detected includes the following steps:

[0066] Using Canny edge detection and the findContours function in OpenCV to find all the contours in the high-noise sketch to be detected, calculating the contour area of each contour, storing the area values in a list, sorting them in descending order to determine the target contour area threshold; next, for the high-noise sketch to be detected, using the area value corresponding to the above target contour as the threshold for finding its final contour, obtaining the final contour of the high-noise sketch to be detected.

[0067] Obtaining the contours of each image in the standard sketch block sequence includes the following steps:

[0068] Read each image in the standard sketch block sequence in turn. Use Canny edge detection and the findContours function in OpenCV to find all the contours in each image in the standard sketch block sequence, and calculate the contour area of each contour. Store the area value corresponding to the target contour in a list. Through ascending sorting, determine that the first element of the list is the minimum threshold, and the last element of the list is the maximum threshold, so as to obtain the contours of each image in the standard sketch block sequence.

[0069] In some embodiments of the present invention, the step of cropping the rectangular area where the contour of the high-noise sketch to be detected is located to obtain the first cropped image includes:

[0070] According to the contour of the high-noise sketch to be detected, obtain the coordinates of the upper left corner vertex of the circumscribed rectangle corresponding to the contour of the high-noise sketch to be detected and the length and width of the circumscribed rectangle;

[0071] According to the coordinates of the upper left corner vertex, length and width of the circumscribed rectangle, crop the high-noise sketch to be detected within the circumscribed rectangle area through slicing to obtain the first cropped image.

[0072] In some embodiments of the present invention, the neural network model is a PP-OCR neural network model.

[0073] In a specific embodiment of the present invention, for the first cropped image, use the SIFT algorithm to create a sift object, and use the detectAndCompute() function to find the key point descriptor information of the first cropped image; for each image in the second cropped image, obtain the key point descriptor information of each image, and save the key point descriptor information of each image as a file in npy format, so as to obtain the corresponding feature dataset of the standard sketch block sequence.

[0074] In some embodiments of the present invention, the step of matching the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped image in turn to obtain a matching result, and obtaining the standard sketch block of the high-noise sketch to be detected according to the matching result includes:

[0075] Input the key point descriptor information of the first cropped image and the key point descriptor information of each image in the second cropped image into the FLANN matcher;

[0076] Search for the first image with the highest matching success rate between the key point descriptor information of each image in the second cropped image and the key point descriptor information of the first cropped image;

[0077] Use the first image as the standard sketch block of the high-noise sketch to be detected.

[0078] In some embodiments of the present invention, the step of searching for the first image with the highest matching success rate between the key point descriptor information of each image in the second cropped image and the key point descriptor information of the first cropped image includes:

[0079] Respectively obtain the ratio of the minimum Euclidean distance to the sub-minimum Euclidean distance between the key points of the first cropped image and the key points of each image in the second cropped image;

[0080] Exclude the corresponding images with the ratio greater than the first preset value from the second cropped image;

[0081] Set the minimum number of matches between the key points of the first cropped image and the key points of each image in the second cropped image, and exclude the incorrect key point matches in the second cropped image according to the findHomography function in OpenCV and the minimum number of matches to obtain a third cropped image;

[0082] Use the image in the third cropped image with the largest number of matching key points with the key points of the first cropped image as the first image.

[0083] In a specific embodiment of the present invention, set a FLANN matcher, select the k-means tree algorithm, and traverse 50 times; then, traverse the feature data set, input the key point descriptor information of each image in the second cropped image and the key point descriptor data of the first cropped image into the FLANN matcher to obtain the matching results with the minimum and sub-minimum Euclidean distances, and then calculate the ratio of the minimum Euclidean distance to the sub-minimum Euclidean distance. If the ratio is greater than 0.4, it is considered an incorrect match, and at the same time, save the correct matching results in a list. <##ID=

[0084] Set the minimum number of matches as the lowest matching reference standard, align the feature points, perform coordinate transformation and convert them to float32 type, use the cv2.findHomography() function, input the RANSAC algorithm in OpenCV, obtain a 3×3 transformation matrix and a mask, and exclude the remaining incorrect matches.

[0085] Next, save the file names of the feature datasets and the corresponding number of successfully matched points in the form of key-value pairs in a dictionary, traverse the dictionary, and obtain the maximum matching result, which is the corresponding standard sketch block in the feature dataset, to obtain the matching result map of the to-be-detected high-noise-removed sketch and the corresponding images in the standard sketch block sequence.

[0086] To better implement a high-noise sketch image matching method in an embodiment of the present invention, correspondingly, based on a high-noise sketch image matching method, as Figure 2 shown, an embodiment of the present invention further provides a high-noise sketch image matching device. A high-noise sketch image matching device 200 includes:

[0087] A mask image acquisition unit 201, configured to input a to-be-detected high-noise sketch into a trained neural network model, and obtain a mask image of the to-be-detected high-noise sketch through the trained neural network model;

[0088] An image denoising unit 202, configured to remove noise in the to-be-detected high-noise sketch based on an image inpainting algorithm, a filtering algorithm, and the mask image;

[0089] An image preprocessing unit 203, configured to perform image grayscale conversion and binarization processing on both the to-be-detected high-noise sketch and a standard sketch sequence, respectively obtaining a processed high-noise sketch and a processed standard sketch sequence;

[0090] An image contour acquisition unit 204, configured to respectively determine the contour of the to-be-detected high-noise sketch and the contour of the standard sketch sequence based on a contour extraction algorithm, the processed high-noise sketch, and the processed standard sketch sequence;

[0091] A high-noise sketch key point information acquisition unit 205, configured to crop a rectangular area where the contour of the to-be-detected high-noise sketch is located to obtain a first cropped image, and determine key point descriptor information of the first cropped image based on the SIFT algorithm;

[0092] A standard sketch sequence key point information acquisition unit 206, configured to respectively crop rectangular areas where the contours of the standard sketch sequence are located to obtain second cropped images, and determine key point descriptor information of each image in the second cropped images based on the SIFT algorithm;

[0093] A standard sketch block acquisition unit 207, configured to sequentially match the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images to obtain a matching result, and obtain a standard sketch block of the to-be-detected high-noise sketch according to the matching result.

[0094] The high-noise sketch image matching device 200 provided by the above embodiments can implement the technical solutions described in the embodiments of the high-noise sketch image matching method. For the specific implementation principles of the above modules or units, reference can be made to the corresponding content in the embodiments of the high-noise sketch image matching method, which will not be elaborated here.

[0095] As Figure 3 shown, the present invention also correspondingly provides an electronic device 300. The electronic device 300 includes a processor 301, a memory 302, and a display 303. Figure 3 Only some components of the electronic device 300 are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0096] In some embodiments, the processor 301 may be a central processing unit (CPU), a microprocessor, or other data processing chips, and is used to run the program code stored in the memory 302 or process data, such as a high-noise sketch image matching method in the present invention.

[0097] In some embodiments, the processor 301 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 301 may be local or remote. In some embodiments, the processor 301 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination of the above.

[0098] In some embodiments, the memory 302 may be an internal storage unit of the electronic device 300, such as a hard disk or memory of the electronic device 300. In some other embodiments, the memory 302 may also be an external storage device of the electronic device 300, such as a plug-in hard disk equipped on the electronic device 300, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0099] Furthermore, the memory 302 may also include both the internal storage unit and the external storage device of the electronic device 300. The memory 302 is used to store the application software installed in the electronic device 300 and various types of data.

[0100] In some embodiments, the display 303 may be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 303 is used to display information of the electronic device 300 and to display a visual user interface. Components 301-303 of the electronic device 300 communicate with each other via a system bus.

[0101] In one embodiment, when the processor 301 executes a high-noise sketch image matching program in the memory 302, the following steps may be implemented:

[0102] Input the high-noise sketch to be detected into a trained neural network model, and obtain a mask image of the high-noise sketch to be detected through the trained neural network model;

[0103] Remove the noise in the high-noise sketch to be detected based on an image inpainting algorithm, a filtering algorithm, and the mask image;

[0104] Perform image grayscale conversion and binarization processing on both the high-noise sketch to be detected and the standard sketch sequence, respectively obtaining a processed high-noise sketch and a processed standard sketch sequence;

[0105] Based on a contour extraction algorithm and the processed high-noise sketch and the processed standard sketch sequence, respectively determine the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence;

[0106] Crop the rectangular area where the contour of the high-noise sketch to be detected is located to obtain a first cropped image, and determine the key point descriptor information of the first cropped image based on the SIFT algorithm;

[0107] Crop the rectangular areas where the contours of the standard sketch sequence are located to obtain second cropped images, and determine the key point descriptor information of each image in the second cropped images based on the SIFT algorithm;

[0108] Match the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images in sequence to obtain a matching result, and obtain the standard sketch block of the high-noise sketch to be detected according to the matching result.

[0109] It should be understood that when the processor 301 executes a high-noise sketch image matching program in the memory 302, in addition to the above functions, other functions may also be implemented. For details, refer to the description of the corresponding method embodiments above.

[0110] Furthermore, the embodiments of the present invention do not specifically limit the type of the mentioned electronic device 300. The electronic device 300 can be a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop, or other portable electronic devices. Exemplary embodiments of the portable electronic device include, but are not limited to, portable electronic devices running IOS, android, microsoft, or other operating systems. The above-mentioned portable electronic device can also be other portable electronic devices, such as a laptop with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 300 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0111] Those skilled in the art can understand that all or part of the processes for implementing the methods of the above embodiments can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium is a disk, an optical disk, a read-only memory, or a random access memory, etc.

[0112] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for matching high-noise sketch images, characterized in that, Including: Input the high-noise sketch to be detected into a trained neural network model, and obtain the mask image of the high-noise sketch to be detected through the trained neural network model; Remove the noise in the high-noise sketch to be detected based on the image inpainting algorithm, filtering algorithm, and the mask image; Perform image grayscaling and binarization processing on both the high-noise sketch to be detected and the standard sketch sequence, and obtain the processed high-noise sketch and the processed standard sketch sequence respectively; Based on the contour extraction algorithm, as well as the processed high-noise sketch and the processed standard sketch sequence, determine the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence respectively; Crop the rectangular area where the contour of the high-noise sketch to be detected is located to obtain a first cropped image, and determine the key point descriptor information of the first cropped image based on the SIFT algorithm; Crop the rectangular areas where the contours of the standard sketch sequence are located respectively to obtain second cropped images, and determine the key point descriptor information of each image in the second cropped images based on the SIFT algorithm; Match the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images in sequence to obtain a matching result, and obtain the standard sketch block of the high-noise sketch to be detected according to the matching result.

2. The high-noise sketch image matching method according to claim 1, characterized in that, The removing the noise in the high-noise sketch to be detected based on the image inpainting algorithm and filtering algorithm includes: Remove the numbers and texts in the high-noise sketch to be detected based on the image inpainting algorithm; Remove the numbers and texts in the high-noise sketch to be detected based on the Gaussian filtering algorithm.

3. A high-noise sketch image matching method according to claim 1, characterized in that The respectively determining the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence based on the contour extraction algorithm includes: Determine the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence based on the Canny algorithm and the findContours function in OpenCV.

4. A high-noise sketch image matching method according to claim 1, characterized in that The cropping the rectangular area where the contour of the high-noise sketch to be detected is located to obtain a first cropped image includes: According to the contour of the high-noise sketch to be detected, obtain the coordinates of the upper left vertex of the circumscribed rectangle corresponding to the contour of the high-noise sketch to be detected, and the length and width of the circumscribed rectangle; Crop the high-noise sketch to be detected within the circumscribed rectangle area through slicing according to the coordinates of the upper left vertex of the circumscribed rectangle, the length, and the width to obtain a first cropped image.

5. A high-noise sketch image matching method according to claim 1, characterized in that The neural network model is the PP-OCR neural network model.

6. A high-noise sketch image matching method according to claim 1, characterized in that The matching the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images in sequence to obtain a matching result, and obtaining the standard sketch block of the high-noise sketch to be detected according to the matching result includes: Input the key point descriptor information of the first cropped image and the key point descriptor information of each image in the second cropped images into a FLANN matcher; In each image key point descriptor information of the second cropped image, find the first image with the highest matching success rate with the key point descriptor information of the first cropped image; Use the first image as the standard sketch block of the high-noise sketch to be detected.

7. A high-noise sketch image matching method according to claim 6, characterized in that, The step of finding the first image with the highest matching success rate with the key point descriptor information of the first cropped image in each image key point descriptor information of the second cropped image includes: Respectively obtain the ratio of the minimum Euclidean distance to the second minimum Euclidean distance between the key points of the first cropped image and the key points of each image in the second cropped image; Remove the corresponding images with the ratio greater than the first preset value from the second cropped image; Set the minimum number of matches between the key points of the first cropped image and the key points of each image in the second cropped image, and remove the incorrect key point matches in the second cropped image according to the findHomography function in OpenCV and the minimum number of matches to obtain a third cropped image; Use the image in the third cropped image with the largest number of matching key points with the key points of the first cropped image as the first image.

8. A high-noise sketch image matching device, characterized in that, including: A mask image acquisition unit for inputting the high-noise sketch to be detected and a trained neural network model, and obtaining a mask image of the high-noise sketch to be detected through the trained neural network model; An image denoising unit for removing noise in the high-noise sketch to be detected based on an image inpainting algorithm, a filtering algorithm, and the mask image; An image preprocessing unit for performing image grayscaling and binarization processing on both the high-noise sketch to be detected and the standard sketch sequence to obtain a processed high-noise sketch and a processed standard sketch sequence respectively; An image contour acquisition unit for respectively determining the contour of the high-noise sketch to be detected and the contour of the standard sketch sequence based on a contour extraction algorithm, the processed high-noise sketch, and the processed standard sketch sequence; A high-noise sketch key point information acquisition unit for cropping the rectangular area where the contour of the high-noise sketch to be detected is located to obtain a first cropped image, and determining the key point descriptor information of the first cropped image based on the SIFT algorithm; A standard sketch sequence key point information acquisition unit for respectively cropping the rectangular areas where the contours of the standard sketch sequence are located to obtain second cropped images, and determining the key point descriptor information of each image in the second cropped images based on the SIFT algorithm; A standard sketch block acquisition unit for sequentially matching the key point descriptor information of the first cropped image with the key point descriptor information of each image in the second cropped images to obtain a matching result, and obtaining the standard sketch block of the high-noise sketch to be detected according to the matching result.

9. An electronic device, characterized in that, including a memory and a processor, wherein, The memory is used for storing programs; The processor is coupled to the memory and is used for executing the program stored in the memory to implement the steps in any one of the above-mentioned high-noise sketch image matching methods as claimed in claims 1 to 7.

10. A computer-readable storage medium, characterized in that, For storing computer-readable programs or instructions, when the programs or instructions are executed by a processor, they can implement the steps in a high-noise sketch image matching method described in any one of claims 1 to 7 above.

Images