An image semantic matching method

Abstract

The present application relates to the field of image matching, and more particularly to a kind of receiving module in image semantic matching method obtains first image;First extraction module extracts several first image feature map, second feature map;Second extraction module obtains several first text feature map, first symbol feature map;Third extraction module obtains several second image feature map, second text feature map and second symbol feature map;First matching module matches several feature maps with storage unit respectively;First classification module outputs several fourth text feature map, fourth symbol feature map;Second classification module outputs several fourth image feature map, fifth text feature map, fifth symbol feature map;Fourth extraction module outputs fifth image feature map, sixth text feature map and sixth symbol feature map;Second matching module filters out several third target image;Determination module determines final target image.The present application is screened by image, text and symbol respectively, so that the matching result is accurate.

Description

Technical Field

[0001] This invention relates to the field of image matching, and more particularly to an image semantic matching method. Background Technology

[0002] Image matching was first proposed in the United States in the 1970s for applications in aircraft auxiliary navigation systems and terminal guidance systems for weapon delivery systems. Since the 1980s, its applications have gradually expanded from purely military applications to other fields. With the development of science and technology, image matching technology has become an extremely important technology in the field of modern information processing, with wide and practical applications in many areas, such as pattern recognition, automatic navigation, medical diagnosis, computer vision, 3D image reconstruction, and remote sensing image processing. Image matching is a bottleneck problem in these application areas, and many important studies in computer vision are currently conducted under the assumption that the matching problem has been solved. Therefore, further in-depth research on image matching is of great significance.

[0003] Patent document CN113255829B discloses a method and apparatus for zero-shot image target detection based on deep learning. The method involves providing an image to be detected and target description text; inputting the image to be detected and the target description text into a text semantic guided detection model to obtain the target detection result output by the text semantic guided detection model; the text semantic guided detection model is derived from a tile semantic guided detection model, which is trained based on an image cropping recall method; the text semantic features encoded by the text semantic guided detection model are matched with the tile semantic features encoded by the tile semantic guided detection model, and the description text corresponding to the text semantic features and the tile representation corresponding to the tile semantic features represent the same target.

[0004] Existing image semantic matching methods, when performing simultaneous text and image matching, can lead to inaccurate image matching results. Summary of the Invention

[0005] To address this issue, the present invention provides an image semantic matching method that solves the problem of inaccurate image matching caused by simultaneously matching images, text, and symbols in an image with images in a target image library during the actual image semantic matching process.

[0006] To achieve the above objectives, the present invention provides an image semantic matching method, which includes a receiving module receiving an input original image and performing noise reduction processing on the original image to obtain a first image;

[0007] The first extraction module extracts a number of first image feature maps and a number of second image feature maps from the first image based on the region area;

[0008] The second extraction module extracts several second feature maps based on effective pixel values ​​to obtain several first text feature maps and several first symbol feature maps;

[0009] The third extraction module further extracts the first image feature map to obtain several second image feature maps, several second text feature maps, and several second symbol feature maps;

[0010] The first matching module matches several second image feature maps, several first text feature maps, several first symbol feature maps, several second text feature maps, and several second symbol feature maps with the first image library, the first text library, and the first symbol library, respectively. If the matching is successful, it outputs several third image feature maps, several third text feature maps, and several third symbol feature maps. If the matching is unsuccessful, it outputs the first output feature map.

[0011] The first classification module further classifies the first output feature map and outputs several fourth text feature maps and several fourth symbol feature maps;

[0012] The second classification module sorts the aforementioned third image feature maps, fourth text feature maps, and fourth symbol feature maps according to their features and outputs them as several fourth image feature maps, several fifth text feature maps, and several fifth symbol feature maps.

[0013] The fourth extraction module extracts the fifth image feature map, the sixth text feature map, and the sixth symbol feature map that are respectively located at the first position among the fourth image feature map, the fifth text feature map, and the fifth symbol feature map;

[0014] The second matching module matches the fifth image feature map with the target image library to select a number of first target images, matches the sixth text feature map with a number of first target images to select a number of second target images, and matches the sixth symbol feature map with a number of second target images to select a number of third target images.

[0015] The determination module compares the similarity between the first image and several of the third target images to determine the final target image.

[0016] Furthermore, the first detection unit in the first extraction module detects the edge contours of continuous regions in the first image and extracts the regions contained in the different edge contours to obtain several first detection images.

[0017] The first calculation unit calculates the area of ​​several of the first detected images to obtain several first area values;

[0018] The first classification unit sets a first area standard value, takes several continuous regions whose first area values ​​are greater than the first area standard value as several first image feature maps, and takes several continuous regions whose first area values ​​are less than the first area standard value as several second feature maps.

[0019] The first extraction unit extracts several first image feature maps and several second feature maps.

[0020] Furthermore, the second detection unit in the second extraction module detects the second feature map to obtain several first effective pixel values ​​of several second feature maps;

[0021] The second classification unit sets a first effective pixel standard value, and takes several continuous regions corresponding to the first effective pixel values ​​that are greater than the first effective pixel standard value as several first text feature maps, and takes several continuous regions corresponding to the first effective pixel values ​​that are less than the first effective pixel standard value as several first symbol feature maps.

[0022] The second extraction unit extracts several of the first text feature maps and several of the first symbol feature maps.

[0023] Furthermore, the third detection unit in the third extraction module performs effective pixel detection on several first image feature maps to obtain several second effective pixel values;

[0024] The third classification unit sets a second effective pixel standard value, takes several continuous regions corresponding to second effective pixel values ​​greater than the second effective pixel standard value as several second text feature maps, and takes several continuous regions corresponding to second effective pixel values ​​less than the second effective pixel standard value as several second symbol feature maps.

[0025] The third extraction unit extracts several second text feature maps and several second symbol feature maps, and the extracted several first image feature maps become several second image feature maps.

[0026] Furthermore, the first storage unit in the first matching module stores a first image library, a first character library, and a first symbol library;

[0027] The first matching unit performs corresponding matching between a number of second image feature maps, a number of first text feature maps, a number of first symbol feature maps, a number of second text feature maps, and a number of second symbol feature maps and the storage unit. If there is a matching result in the storage unit, it outputs a number of third image feature maps, a number of third text feature maps, and a number of third symbol feature maps. If there is no matching result in the storage unit, it outputs a number of first output feature maps for reclassification.

[0028] Furthermore, the fourth detection unit in the first classification module performs region contour detection on several of the first output feature maps;

[0029] The second calculation unit performs area calculations on several first output feature maps and outputs several first output area values.

[0030] The fourth classification unit sets a second area standard value, and takes several continuous regions whose first output area value is greater than the second area standard value as several fourth text feature maps, and several continuous regions whose first output area value is less than the second area standard value as several fourth symbol feature maps.

[0031] The fourth extraction unit extracts several of the fourth text feature maps and several of the fourth symbol feature maps.

[0032] Furthermore, the first receiving unit in the second classification module receives several third image feature maps, several fourth text feature maps, and several fourth symbol feature maps;

[0033] The third calculation unit calculates the area of ​​the regions of the third image features and outputs a number of second output area values, calculates the effective pixel values ​​of the regions of the fourth text feature maps and outputs a number of third effective pixel values, and calculates the effective pixel values ​​of the regions of the fourth symbol feature maps and outputs a number of fourth effective pixel values.

[0034] The fifth classification unit sorts the second output area values, the third effective pixel values, and the fourth effective pixel values ​​from largest to smallest, respectively.

[0035] The fifth extraction unit outputs the sorted feature images as several fourth image feature maps, several fifth text feature maps, and several fifth symbol feature maps.

[0036] Furthermore, the fourth extraction module extracts the first feature image of several fourth image feature maps and outputs it as a fifth image feature map, extracts the first feature image of several fifth text feature maps and outputs it as a sixth text feature map, and extracts the first feature image of several fifth symbol feature maps and outputs it as a sixth symbol feature map.

[0037] Furthermore, the second storage unit in the second matching module stores the target image library;

[0038] The second matching unit performs contour matching between the fifth image feature map and the target image library to filter out a number of first target images;

[0039] The third matching unit matches the sixth text feature map with several first target images based on the effective pixel values ​​to filter out several second target images;

[0040] The fourth matching unit matches the sixth symbol feature map with several second target images based on the effective pixel values ​​to filter out several third target images;

[0041] The sixth extraction unit extracts several of the third target images.

[0042] Furthermore, the fourth calculation unit in the determining module calculates the similarity between the first image and several of the third target images;

[0043] The sixth classification unit sorts the similarity scores and their corresponding target images and outputs the fourth target image.

[0044] The seventh extraction unit extracts the first image from the fourth target image as the final target image.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention performs noise reduction processing on the original image through the receiving module, making the original image clearer and facilitating feature extraction from the first image; the first extraction module, the second extraction module, and the third extraction module extract several preliminary feature maps of the image, text, and symbols from the first image, thereby achieving segmentation of the first image, so that the image matching process is determined by the image, text, and symbols respectively; the first matching module checks the preliminary feature maps of the image, text, and symbols to determine the correctness of the preliminary feature maps; and the first classification module further classifies the preliminary feature maps to obtain several final feature maps, thus segmenting the image, text, and symbols. The feature maps of characters and symbols are used to verify the accuracy of the classification results. Inaccurate classifications are reclassified to improve accuracy and facilitate subsequent matching. The second classification module and the fourth extraction module extract the final feature maps to obtain the top-ranked feature maps. These top-ranked feature maps are then matched with the target image library to determine the third target images. The second matching module sequentially filters the target image library based on image matching, text matching, and symbol matching to ensure accurate matching. Finally, the determination module compares similarity to determine the final target image, achieving accurate matching.

[0046] In particular, the first detection unit obtains several first detection images through the edge contours of continuous regions, thereby simplifying the processing of the first image and improving the efficiency of processing the first image. The first calculation unit calculates the area of ​​several first detection images through the contours of several first detection images, making the calculation results accurate and highly precise. The first classification unit sets the first area standard value to distinguish between the image and the text symbol, thereby achieving the effect of distinguishing several first image feature maps and several second feature maps.

[0047] In particular, the second classification unit distinguishes the second feature map by setting the first effective pixel standard value to obtain several first text feature maps and several first symbol feature maps, thereby achieving the effect of distinguishing between text and symbols. By distinguishing between text and symbols through the first effective pixel standard value, it facilitates the operation of text and symbol recognition and extraction, thereby improving the efficiency of image processing.

[0048] In particular, the third detection unit distinguishes the text and symbol features contained in several first image feature maps by detecting the effective pixel values ​​of several first image feature maps, and distinguishes text and symbols by the second effective pixel standard value, thereby maintaining the clarity of text and symbols and improving the efficiency of image processing.

[0049] In particular, the first matching unit performs corresponding matching between a number of second image feature maps, a number of first text feature maps, a number of first symbol feature maps, a number of second text feature maps, and a number of second symbol feature maps and the storage unit to verify the feature maps of the image, text, and symbols, thereby achieving a more accurate effect on the feature maps of the image, text, and symbols and facilitating subsequent image processing.

[0050] In particular, the fourth classification unit distinguishes several first output feature maps using the second area standard value, and reclassifies the text and symbol feature maps that cannot be matched in the storage unit, thereby achieving accurate classification and improving the efficiency of subsequent image recognition processing. The second area standard value is used to distinguish text and symbols again when the effective pixel value cannot be distinguished.

[0051] In particular, the third calculation unit achieves the effect of distinguishing the proportion of image features by calculating the area of ​​several third image features, and achieves the distinction between different text and symbol features by calculating the effective pixel values ​​of several fourth text feature maps and several fourth symbol feature maps, so as to facilitate subsequent image processing.

[0052] In particular, the fourth extraction module extracts the first and second digits of several fourth image feature maps, several fifth text feature maps, and several fifth symbol feature maps respectively to extract the feature maps that are at the optimal solution among the image, text, and symbol features, which facilitates subsequent image matching and improves image matching efficiency.

[0053] In particular, the initial screening process of the target image library is achieved by using the fifth image feature map to perform contour matching with the target image library to select several first target images. The second screening process of the target image library is achieved by using the sixth text feature map to match with several first target images based on effective pixel values. The third screening process of the target image library is achieved by using the sixth symbol feature map to match with several second target images based on effective pixel values ​​to select several third target images. By matching and screening the target image library with images, text, and symbols respectively, the matching process achieves an accurate effect.

[0054] In particular, by performing a final screening of the plurality of third target images based on the similarity, a detailed comparison and analysis of various features of the first image and the plurality of third target images is conducted to achieve an accurate matching result between the final target image and the first image, thereby improving the matching accuracy. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating the image semantic matching method provided in an embodiment of the present invention;

[0056] Figure 2 This is a first functional block diagram of the image semantic matching method provided in an embodiment of the present invention;

[0057] Figure 3 This is a second functional block diagram of the image semantic matching method provided in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0059] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0060] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0061] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0062] Please see Figure 1 and Figure 2 As shown, an embodiment of the present invention provides an image semantic matching method, which includes: step S110, receiving module 10 receives the input original image and performs noise reduction processing on the original image to obtain a first image;

[0063] Step S120: The first extraction module 20 extracts the first image based on the area of ​​the region to obtain several first image feature maps and several second feature maps;

[0064] Step S130: The second extraction module 30 extracts several second feature maps based on effective pixel values ​​to obtain several first text feature maps and several first symbol feature maps;

[0065] Step S140: The third extraction module 40 extracts the first image feature map again to obtain several second image feature maps, several second text feature maps, and several second symbol feature maps.

[0066] In step S150, the first matching module 50 matches several second image feature maps, several first text feature maps, several first symbol feature maps, several second text feature maps and several second symbol feature maps with the first image library, the first text library and the first symbol library respectively. If the matching is successful, several third image feature maps, several third text feature maps and several third symbol feature maps are output. If the matching is unsuccessful, the first output feature map is output.

[0067] Step S160: The first classification module 60 reclassifies the first output feature map and outputs several fourth text feature maps and several fourth symbol feature maps.

[0068] In step S170, the second classification module 70 sorts the several third image feature maps, several fourth text feature maps, and several fourth symbol feature maps according to their features and outputs several fourth image feature maps, several fifth text feature maps, and several fifth symbol feature maps.

[0069] Step S180: The fourth extraction module 80 extracts the fifth image feature map, the sixth text feature map, and the sixth symbol feature map that are respectively located at the first position in the plurality of the fourth image feature maps, the plurality of the fifth text feature maps, and the plurality of the fifth symbol feature maps;

[0070] In step S190, the second matching module 90 matches the fifth image feature map with the target image library to select a number of first target images, matches the sixth text feature map with a number of first target images to select a number of second target images, and matches the sixth symbol feature map with a number of second target images to select a number of third target images.

[0071] In step S200, the determining module 100 compares the similarity between the first image and several of the third target images to determine the final target image.

[0072] Specifically, the receiving module 10 performs noise reduction processing on the original image to make it clearer and facilitate image extraction. The first extraction module 20, the second extraction module 30, and the third extraction module 40 extract several preliminary feature maps of the image, text, and symbols from the first image. The first matching module 50 checks the preliminary feature maps of the image, text, and symbols to determine their correctness. The first classification module 60 further classifies the preliminary feature maps to obtain several final feature maps. The second classification module 70 and the fourth extraction module 80 extract the final feature maps to obtain the first-ranked feature map among the final feature maps. The second matching module 90 and the determining module 100 determine the final target image to achieve accurate matching.

[0073] See Figure 2As shown, specifically, in the first extraction module 20, the first detection unit 21 detects the edge contours of continuous regions in the first image and extracts the regions contained in the different edge contours to obtain a plurality of first detection images; the first calculation unit 22 calculates the area of ​​the plurality of first detection images to obtain a plurality of first area values; the first classification unit 23 sets a first area standard value, and takes the corresponding continuous regions with the first area values ​​greater than the first area standard value as a plurality of first image feature maps, and takes the corresponding continuous regions with the first area values ​​less than the first area standard value as a plurality of second feature maps; the first extraction unit 24 extracts the plurality of first image feature maps and the plurality of second feature maps.

[0074] Specifically, the first detection unit 21 obtains several first detection images through the edge contours of continuous regions to simplify the processing of the first image and improve the efficiency of processing the first image. The first calculation unit 22 calculates the area of ​​several first detection images through the contours of several first detection images to make the calculation results accurate and highly precise. The first classification unit 23 sets the first area standard value to distinguish the image and text symbols, thereby achieving the effect of distinguishing several first image feature maps and several second feature maps.

[0075] Specifically, the second detection unit 31 in the second extraction module 30 detects the second feature map to obtain several first effective pixel values ​​of several second feature maps; the second classification unit 32 sets a first effective pixel standard value, takes several continuous regions corresponding to several first effective pixel values ​​greater than the first effective pixel standard value as several first text feature maps, and takes several continuous regions corresponding to several first effective pixel values ​​less than the first effective pixel standard value as several first symbol feature maps; the second extraction unit 33 extracts several first text feature maps and several first symbol feature maps.

[0076] Specifically, the second classification unit 32 distinguishes the second feature map by setting the first effective pixel standard value to obtain several first text feature maps and several first symbol feature maps, thereby achieving the effect of distinguishing between text and symbols. By distinguishing between text and symbols through the first effective pixel standard value, it facilitates the operation of text and symbol recognition and extraction, thereby improving the efficiency of image processing.

[0077] Specifically, the third detection unit 41 in the third extraction module 40 performs effective pixel detection on several first image feature maps to obtain several second effective pixel values; the third classification unit 42 sets a second effective pixel standard value, and takes several continuous regions corresponding to several second effective pixel values ​​greater than the second effective pixel standard value as several second text feature maps, and takes several continuous regions corresponding to several second effective pixel values ​​less than the second effective pixel standard value as several second symbol feature maps; the third extraction unit 43 extracts several second text feature maps and several second symbol feature maps, and the extracted several first image feature maps become several second image feature maps.

[0078] Specifically, the third detection unit 41 distinguishes the text and symbol features contained in several first image feature maps by detecting the effective pixel values ​​of several first image feature maps, and distinguishes the text and symbols by the second effective pixel standard value to maintain the clarity of the text and symbols, thereby improving the efficiency of image processing.

[0079] Specifically, the first storage unit 51 in the first matching module 50 stores a first image library, a first text library, and a first symbol library; the first matching unit 52 performs corresponding matching between a number of second image feature maps, a number of first text feature maps, a number of first symbol feature maps, a number of second text feature maps, and a number of second symbol feature maps and the first storage unit 51. If there is a matching result in the first storage unit 51, it outputs a number of third image feature maps, a number of third text feature maps, and a number of third symbol feature maps. If there is no matching result in the first storage unit 51, it outputs a number of first output feature maps for reclassification.

[0080] Specifically, those skilled in the art will understand that there are no specific requirements for the first image library, the first character library, and the first symbol library, and any one of the image libraries can be selected as the first image library, the first character library, and the first symbol library.

[0081] Specifically, the fourth detection unit 61 in the first classification module 60 performs region contour detection on several first output feature maps; the second calculation unit 62 performs area calculation on several first output feature maps and outputs several first output area values; the fourth classification unit 63 sets a second area standard value, and takes several continuous regions whose first output area values ​​are greater than the second area standard value as several fourth text feature maps, and takes several continuous regions whose first output area values ​​are less than the second area standard value as several fourth symbol feature maps; the fourth extraction unit 64 extracts several fourth text feature maps and several fourth symbol feature maps.

[0082] Specifically, the fourth classification unit 63 distinguishes several first output feature maps using the second area standard value, and reclassifies the text and symbol feature maps that cannot be matched in the first storage unit 51 to achieve accurate classification and improve the efficiency of subsequent image recognition processing. The second area standard value is used to distinguish text and symbols again when the effective pixel value cannot be distinguished.

[0083] Specifically, the first receiving unit 71 in the second classification module 70 receives a plurality of third image feature maps, a plurality of fourth text feature maps, and a plurality of fourth symbol feature maps; the third calculation unit 72 calculates the area of ​​the regions of the plurality of third image features and outputs a plurality of second output area values, calculates the effective pixel values ​​of the regions of the plurality of fourth text feature maps and outputs a plurality of third effective pixel values, and calculates the effective pixel values ​​of the regions of the plurality of fourth symbol feature maps and outputs a plurality of fourth effective pixel values; the fifth classification unit 73 sorts the plurality of second output area values, the plurality of third effective pixel values, and the plurality of fourth effective pixel values ​​from largest to smallest; and the fifth extraction unit 74 outputs the feature images corresponding to the sorting as a plurality of fourth image feature maps, a plurality of fifth text feature maps, and a plurality of fifth symbol feature maps.

[0084] Specifically, the fourth extraction module 80 extracts the first feature image of several fourth image feature maps and outputs it as a fifth image feature map, extracts the first feature image of several fifth text feature maps and outputs it as a sixth text feature map, and extracts the first feature image of several fifth symbol feature maps and outputs it as a sixth symbol feature map.

[0085] Specifically, the second storage unit 91 in the second matching module 90 stores a target image library; the second matching unit 92 performs contour matching between the fifth image feature map and the target image library to filter out a number of first target images; the third matching unit 93 matches the sixth text feature map with a number of first target images based on effective pixel values ​​to filter out a number of second target images; the fourth matching unit 94 matches the sixth symbol feature map with a number of second target images based on effective pixel values ​​to filter out a number of third target images; and the sixth extraction unit 95 extracts a number of the third target images.

[0086] Specifically, the target image library is initially screened by matching the fifth image feature map with the target image library to select several first target images. Then, the target image library is screened by matching the sixth text feature map with the first target images based on the effective pixel values ​​to select several second target images. Finally, the target image library is screened by matching the sixth symbol feature map with the second target images based on the effective pixel values ​​to select several third target images. This process achieves a more accurate matching effect by matching and screening the target image library with images, text, and symbols respectively.

[0087] Specifically, the fourth calculation unit 101 in the determination module 100 calculates the similarity between the first image and several of the third target images; the sixth classification unit 102 sorts the similarity and the corresponding target images and outputs them as the fourth target image; the seventh extraction unit 103 extracts the first image in the fourth target image as the final target image.

[0088] Specifically, by using the similarity to perform final screening of the plurality of third target images, a detailed comparison and analysis of various features of the first image and the plurality of third target images is conducted to achieve accurate matching results between the final target image and the first image, thereby improving matching accuracy.

[0089] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0090] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image semantic matching method, characterized in that, a receiving module receives an input original image and performs denoising processing on the original image to obtain a first image; a first extraction module extracts the first image according to area to obtain a plurality of first image feature maps and a plurality of second feature maps; a second extraction module extracts a plurality of the second feature maps according to effective pixel values to obtain a plurality of first text feature maps and a plurality of first symbol feature maps; a third extraction module re-extracts the first image feature maps to obtain a plurality of second image feature maps, a plurality of second text feature maps and a plurality of second symbol feature maps; a first matching module respectively matches a plurality of the second image feature maps, a plurality of the first text feature maps, a plurality of the first symbol feature maps, a plurality of the second text feature maps and a plurality of the second symbol feature maps with a first image library, a first text library and a first symbol library, and outputs a plurality of third image feature maps, a plurality of third text feature maps, a plurality of third symbol feature maps if the matching is successful, and outputs a first output feature map if the matching is unsuccessful; a first classification module re-classifies the first output feature map to output a plurality of fourth text feature maps and a plurality of fourth symbol feature maps; a second classification module sorts a plurality of the third image feature maps, a plurality of fourth text feature maps and a plurality of fourth symbol feature maps according to features to output a plurality of fourth image feature maps, a plurality of fifth text feature maps and a plurality of fifth symbol feature maps; a fourth extraction module extracts a fifth image feature map, a sixth text feature map and a sixth symbol feature map located at the first position in a plurality of the fourth image feature maps, a plurality of the fifth text feature maps and a plurality of the fifth symbol feature maps, respectively; a second matching module matches the fifth image feature map with a target image library to screen out a plurality of first target images, matches the sixth text feature map with a plurality of the first target images to screen out a plurality of second target images, and matches the sixth symbol feature map with a plurality of the second target images to screen out a plurality of third target images; a determination module compares the similarity of the first image and a plurality of the third target images to determine a final target image.

2. The image semantic matching method of claim 1, wherein, A first detection unit in the first extraction module detects the edge profile of a continuous region in the first image and extracts the region contained by different edge profiles to obtain a plurality of first detection images; A first calculation unit calculates the area of a plurality of the first detection images to obtain a plurality of first area values; A first classification unit sets a first area standard value, and takes the corresponding continuous region with a plurality of the first area values greater than the first area standard value as a plurality of first image feature maps, and takes the corresponding continuous region with a plurality of the first area values less than the first area standard value as a plurality of second feature maps; A first extraction unit extracts a plurality of the first image feature maps and a plurality of the second feature maps.

3. The image semantic matching method of claim 2, wherein, A second detection unit in the second extraction module detects the second feature maps to obtain a plurality of first effective pixel values of a plurality of the second feature maps; The second classification unit sets a first effective pixel standard value, and regards a continuous region corresponding to a first effective pixel value greater than the first effective pixel standard value as a first character feature map, and regards a continuous region corresponding to a first effective pixel value less than the first effective pixel standard value as a first symbol feature map; The second extraction unit extracts the first character feature map and the first symbol feature map.

4. The image semantic matching method of claim 3, wherein, The third detection unit in the third extraction module performs effective pixel detection on the first image feature map to obtain a second effective pixel value; The third classification unit sets a second effective pixel standard value, and regards a continuous region corresponding to a second effective pixel value greater than the second effective pixel standard value as a second character feature map, and regards a continuous region corresponding to a second effective pixel value less than the second effective pixel standard value as a second symbol feature map; The third extraction unit extracts the second character feature map and the second symbol feature map, and the extracted first image feature map is a second image feature map.

5. The image semantic matching method of claim 4, wherein, The first storage unit in the first matching module stores a first image library, a first character library and a first symbol library; The first matching unit matches the second image feature map, the first character feature map, the first symbol feature map, the second character feature map and the second symbol feature map with the storage unit, and if there is a matching result in the storage unit, the matching result is output as a third image feature map, a third character feature map and a third symbol feature map, and if there is no matching result in the storage unit, the matching result is output as a first output feature map for reclassification.

6. The image semantic matching method of claim 5, wherein, The fourth detection unit in the first classification module performs region contour detection on the first output feature map; The second calculation unit calculates the area of the first output feature map to output a first output area value; The fourth classification unit sets a second area standard value, and regards a continuous region corresponding to a first output area value greater than the second area standard value as a fourth character feature map, and regards a continuous region corresponding to a first output area value less than the second area standard value as a fourth symbol feature map; The fourth extraction unit extracts the fourth character feature map and the fourth symbol feature map.

7. The image semantic matching method of claim 6, wherein, The first receiving unit in the second classification module receives the third image feature map, the fourth character feature map and the fourth symbol feature map; The third calculation unit calculates the area of the third image feature map to output a second output area value, calculates the effective pixel value of the fourth character feature map to output a third effective pixel value, and calculates the effective pixel value of the fourth symbol feature map to output a fourth effective pixel value; The fifth classification unit sorts the second output area value, the third effective pixel value and the fourth effective pixel value from large to small respectively. The fifth extraction unit outputs the feature images corresponding to the sorting as a plurality of fourth image feature maps, a plurality of fifth character feature maps and a plurality of fifth symbol feature maps.

8. The image semantic matching method of claim 7, wherein, The fourth extraction module extracts the feature images at the head of the plurality of fourth image feature maps as a fifth image feature map, extracts the feature images at the head of the plurality of fifth character feature maps as a sixth character feature map, and extracts the feature images at the head of the plurality of fifth symbol feature maps as a sixth symbol feature map.

9. The image semantic matching method of claim 8, wherein, The second storage unit in the second matching module stores a target image library; The second matching unit performs contour matching on the fifth image feature map and the target image library to screen a plurality of first target images; The third matching unit performs matching on the sixth character feature map and the plurality of first target images according to effective pixel values to screen a plurality of second target images; The fourth matching unit performs matching on the sixth symbol feature map and the plurality of second target images according to effective pixel values to screen a plurality of third target images; The sixth extraction unit extracts the plurality of third target images.

10. The image semantic matching method of claim 9, wherein, The fourth calculation unit in the determination module calculates the similarity between the first image and the plurality of third target images; The sixth classification unit sorts the similarity and the target image corresponding thereto as a fourth target image; The seventh extraction unit extracts the image at the head of the fourth target image as a final target image.

Images