Target image recognition method based on multi-modal technology

Abstract

This invention relates to a target image recognition method based on multimodal technology, comprising: acquiring a target image to be recognized, and preprocessing it to obtain a target-sized target image to be recognized; constructing a database of images to be matched, the database storing for each target image: a target-sized target image to be matched, a first edge matrix of the target-sized target image to be matched, and a feature map of the target image to be matched; acquiring a second edge matrix of the target-sized target image to be recognized, and filtering initial target images from the target images based on the edge similarity between the second edge matrix and the first edge matrix; acquiring the target image feature map of the target-sized target image to be recognized, and determining the target image to be matched based on the feature similarity between the target image feature map and the initial target image feature map, as the recognition result. This invention performs coarse screening using an edge detection algorithm followed by fine screening based on feature enhancement and feature matrix acquisition, reducing computational load while ensuring recognition efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of multimodal technology, and more specifically to a target image recognition method based on multimodal technology. Background Technology

[0002] With the development of multimodal technology, image recognition technology has been widely applied to many existing application scenarios such as supermarket images, video checkout, and image search on e-commerce platforms.

[0003] Traditional image recognition technologies mostly perform feature matching directly between all images to be matched and the target image. However, feature matching involves a large amount of computation and many feature points, which undoubtedly limits the recognition efficiency.

[0004] In addition, the feature point distribution of different images varies greatly, and the color value transition characteristics are ever-changing. It is impossible to cover all types of images by using only one feature recognition method, which also affects the success rate of image recognition to a certain extent.

[0005] Therefore, there is an urgent need for a new image recognition method that can improve the recognition success rate while reducing the amount of computation. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a target image recognition method based on multimodal technology, comprising:

[0007] Step S1: Obtain the image to be recognized as the target image, and preprocess it to obtain the target-sized image to be recognized;

[0008] Step S2: Construct a database of images to be matched. The database stores the following for each image to be matched: the target size image to be matched, the first edge matrix of the target size image to be matched, and the feature map of the image to be matched.

[0009] Step S3: Obtain the second edge matrix of the target-size image to be identified, and filter the initial images to be matched from the images to be matched based on the edge similarity between the second edge matrix and the first edge matrix;

[0010] Step S4: Obtain the feature map of the image to be identified of the target size image to be identified. Based on the feature similarity between the feature map of the image to be identified and the feature map of the initial image to be matched, determine the matching image that matches the image to be identified, and use it as the recognition result.

[0011] The target size image to be matched is obtained by scaling the image to be matched proportionally and filling it with black pixels.

[0012] The target size image to be identified is obtained by scaling the image to be identified proportionally and filling it with black pixels.

[0013] The method for obtaining the first edge matrix of the target-size image to be matched includes: obtaining the edges of the target-size image to be matched through an edge detection algorithm, and mapping the edges of the target-size image to be matched into a first edge matrix based on the position of the pixel points of the image edges;

[0014] The method for obtaining the second edge matrix of the target-size image to be identified includes: obtaining the edges of the target-size image to be identified through an edge detection algorithm, and mapping the edges of the target-size image to be identified into a second edge matrix based on the position of the pixel points of the image edges.

[0015] The database of images to be matched also stores the first feature matrix, the second feature matrix, and the third feature matrix of the feature maps of the images to be matched.

[0016] Methods for obtaining the feature similarity between the feature map of the image to be identified and the feature map of the initial image to be matched include:

[0017] Determine the first, second, and third feature matrices of the feature map to be identified in the image to be identified.

[0018] A first similarity is determined based on the first feature matrix to be identified and the first feature matrix to be matched in the initial image feature map; a second similarity is determined based on the second feature matrix to be identified and the second feature matrix to be matched in the initial image feature map; a third similarity is determined based on the third feature matrix to be identified and the third feature matrix to be matched in the initial image feature map.

[0019] Feature similarity is determined based on the first similarity, second similarity, and third similarity.

[0020] The method for determining the first feature matrix of the image feature map to be matched includes: dividing the image feature map to be matched into several pixel blocks, determining the first feature point A based on the difference between the pixel at the center point of the pixel block and other pixels in the pixel block, and mapping the first feature matrix to be matched based on the position of the first feature point A.

[0021] The method for determining the first feature matrix of the image feature map to be identified includes: dividing the image feature map to be identified into several pixel blocks, determining the first feature point B based on the difference between the center pixel of the pixel block and other pixels in the pixel block, and mapping the first feature matrix to be identified based on the position of the first feature point B.

[0022] The method for determining the second feature matrix of the image feature map to be matched includes: dividing the image feature map to be matched into several pixel blocks, determining the second feature point A based on the difference between the pixel above and the pixel below the center point of the pixel block, and mapping the second feature matrix based on the position of the second feature point A.

[0023] The method for determining the second feature matrix of the image feature map to be identified includes: dividing the image feature map to be identified into several pixel blocks, determining the second feature point B based on the difference between the pixel above and the pixel below the center point of the pixel block, and mapping the second feature matrix based on the position of the second feature point B.

[0024] The method for determining the third feature matrix of the image feature map to be matched includes: dividing the image feature map to be matched into several pixel blocks, determining the third feature point A based on the difference between the left and right pixels of the center pixel of the pixel block, and mapping the third feature matrix based on the position of the third feature point A.

[0025] The method for determining the third feature matrix of the image feature map to be identified includes: dividing the image feature map to be identified into several pixel blocks, determining the third feature point B based on the difference between the left and right pixels of the center pixel of the pixel block, and mapping the third feature matrix based on the position of the third feature point B.

[0026] The target image recognition method based on multimodal technology of the present invention performs coarse screening through edge detection algorithm and fine screening based on feature enhancement and feature matrix acquisition, thereby reducing the amount of computation while ensuring recognition efficiency and accuracy. Attached Figure Description

[0027] Figure 1 : A schematic diagram of the edge pixel distribution of a target-size image to be matched according to an embodiment of the present invention.

[0028] Figure 2 A feature map pixel block division method according to an embodiment of the present invention.

[0029] Figure 3 Another embodiment of the present invention is a feature map pixel block division method.

[0030] Figure 4 : A schematic diagram of pixel distribution in a pixel block according to the present invention.

[0031] Figure 5 : A schematic diagram of the distribution of the first feature points of the feature map to be matched according to an embodiment of the present invention. Detailed Implementation

[0032] To gain a better understanding of the technical solution and beneficial effects of the present invention, the technical solution of the present invention and its beneficial effects are described in detail below with reference to the accompanying drawings.

[0033] The target image recognition method based on multimodal technology of the present invention includes the following steps S1 to S5.

[0034] Step S1: Obtain the image to be recognized as the target image, and preprocess it to obtain the target size image to be recognized.

[0035] The first step in preprocessing is proportional scaling to avoid image structure distortion. The scaling ratio is determined by the following method:

[0036] If the target size is larger than the original size, then the ratio k = min( , );

[0037] If the target size is smaller than the original size, then the ratio k = max( , );

[0038] in, For the length of the target size, For the target size height, The length of the original image size. The height is the original size of the image.

[0039] In other words, the basic principle of proportional scaling is to ensure that after scaling the original image, at least one of its length or width is consistent with the target size.

[0040] The second step of preprocessing is to fill pixels for either the length or width that does not reach the target size: if the scaled original image does not reach the target size in the length direction, the scaled image is placed at the center of the top and bottom, and filled with black pixels symmetrically in the top and bottom directions; correspondingly, if the scaled original image does not reach the target size in the width direction, the scaled image is placed at the center of the left and right directions, and filled with black pixels symmetrically in the left and right directions. This avoids the image content position shift that may cause subsequent positioning deviations and ensures that the integrity of the image structure is maintained.

[0041] Step S2: Construct a database of images to be matched. The database stores for each image to be matched: the target size image to be matched, the first edge matrix of the target size image to be matched; and the feature map of the image to be matched, the first feature matrix of the feature map to be matched, the second feature matrix of the feature map to be matched, and the third feature matrix of the feature map to be matched.

[0042] The method for obtaining the target size image to be matched is the same as the method for obtaining the target size image to be identified described above, which involves scaling the image proportionally and filling it with black pixels, and will not be repeated here. It should be noted that the target size image to be matched and the target size image to be identified are the same size so that the subsequent comparison benchmark is consistent, which is easily understood by those skilled in the art.

[0043] The method for obtaining the first edge matrix of the target-size image to be matched includes: obtaining the edges of the target-size image to be matched through an edge detection algorithm, and mapping the edges of the target-size image to be matched into the first edge matrix based on the position of the pixel points of the image edges.

[0044] The edge detection algorithm can be selected from any existing technology, such as the Roberts operator, Prewitt operator, Sobel operator, Krisch operator and other edge detection algorithms. This invention does not limit the algorithm.

[0045] The specific method for obtaining the first edge matrix may include: constructing a coordinate system with one corner of the target-sized image to be matched as the origin of the coordinate system, taking one pixel as a coordinate system unit value, mapping each pixel of the image edge to a point in the coordinate system, and for each point in the coordinate system, taking the value as 1 when there is an edge pixel at the point, and taking the value as 0 when there is no edge pixel at the point, thus obtaining the first edge matrix.

[0046] by Figure 1 For example, taking the bottom left corner of the target-size image to be matched as the point (0, 0) in the coordinate system, the size of this image corresponds to 6 pixels in both the horizontal and vertical axes. Figure 1 In the image, the pixels filled with black are those involved in the image edges. Figure 1 The image edges in the image can be mapped to the following first edge matrix:

[0047] .

[0048] The feature map of the image to be matched is obtained by a multi-round feature map generation unit that extracts features and upscales the target size image to be matched. Each round of feature map generation unit includes a first branch and a second branch. The first branch performs channel feature normalization, nonlinear transformation, convolutional layer feature extraction, and size upscaling on the input image. The second branch performs resolution upscaling on the input image. Then, the feature maps processed by the first and second branches are fused together, and the fused feature map is output to the next feature map generation unit. The feature map output by the last round of feature map generation unit is the feature map of the image to be matched.

[0049] That is, the first branch focuses on extracting detailed features, and the second branch focuses on improving image resolution. After multiple rounds of feature map generation units, the final generated feature map to be matched is a feature map that has been refined and improved in resolution. In subsequent feature matching with the image to be identified, it is easier to mine detailed features. Therefore, the number of pixel blocks during feature matching can be appropriately reduced (detailed later), thus reducing the matrix dimension (detailed later) and the amount of computation.

[0050] The methods for obtaining the first, second, and third feature matrices of the image to be matched are all to divide the image feature map into several pixel blocks and then identify feature pixels from the center pixel of each pixel block. (The following is a continuation of the previous sentence.) Figure 2 and Figure 3 The pixel block division method of the present invention is described in detail.

[0051] Please see Figure 2 In one embodiment, the edges of each pixel block are adjacent. Figure 2 In the image, different shapes of shadow fills represent a pixel block. Each pixel block involves 9 pixels. An initial pixel block can be created by randomly selecting one of the four vertices of the image, and then dividing the image into 9-pixel blocks. This process continues until there is insufficient space at the image edges. Figure 2 The parts that are not filled with shadows can be discarded directly.

[0052] Please see Figure 3 In one embodiment, the pixel blocks are divided using a sliding window method. That is, different pixel block regions overlap. Specifically, during the division, each pixel block is shifted one pixel unit horizontally or vertically relative to its adjacent pixel blocks. Figure 3 Each pixel block in the image involves 9 pixels, with the center pixels of each block adjacent to each other.

[0053] The first pixel block is surrounded by red borders on the left and right and black borders on the top and bottom, with the center pixel being point A.

[0054] The second pixel block is slid one pixel unit to the right from the first pixel block, and is surrounded by blue borders on the left and right and black borders on the top and bottom, with point B as the center pixel.

[0055] The third pixel block slides down one pixel from the first pixel block, with the center pixel point being point C, and so on.

[0056] exist Figure 2 Under the pixel block division method, all pixel blocks can be used for subsequent feature matrix calculation; Figure 3 Under the pixel block division method, several pixel blocks can be randomly selected for subsequent feature matrix calculation.

[0057] The method for determining the first feature matrix to be matched in the feature map of the image to be matched is as follows: For the extracted pixel block, calculate the difference between the center pixel of each pixel block and other pixels in the pixel block, determine the first feature point from the center pixel of each pixel block based on the difference, and map the first feature matrix to be matched based on the position of the first feature point.

[0058] Please combine Figure 4As shown, it displays a pixel block containing pixels A0, A1, A2, A3, A4, A5, A6, A7, and A8, with pixel A0 as the center pixel. Then:

[0059] C1= ;

[0060] Where C1 is the sum of the differences between the center pixel A0 and the other pixels. The three-channel color values ​​of the center pixel A0, The three-channel color values ​​of other pixels are represented by 'i', which ranges from 0 to 8.

[0061] The calculated C1 value is compared with the first threshold. When it is greater than the first threshold, the center pixel is determined as the first feature point. Then, similar to the method for determining the first edge matrix, a coordinate system is constructed with one corner of the feature map of the image to be matched as the origin of the coordinate system. One pixel is used as a coordinate system unit value. Each first feature point is mapped to a point in the coordinate system. For each point in the coordinate system, when there is a first feature point at the point, the value is taken as the difference C1 corresponding to the first feature point. When there is no first feature point, the value is taken as 0, thus obtaining the first feature matrix to be matched.

[0062] like Figure 5 As shown, with the lower left corner of the feature map of the image to be matched as the point (0, 0) in the coordinate system, the image corresponds to 7 pixels in both the horizontal and vertical directions. Figure 5 In the diagram, the pixels filled with numbers are the first feature points, and the specific numbers are the differences C1 corresponding to the first feature points. Figure 5 The first feature point in the matrix can be mapped to the following first feature matrix to be matched:

[0063] .

[0064] The method for determining the second feature matrix to be matched in the feature map of the image to be matched is as follows: For the extracted pixel block, calculate the difference between the pixel above and the pixel below the center pixel of each pixel block, determine the second feature point from the center pixel of each pixel block based on the difference, and map the second feature matrix to be matched based on the position of the second feature point.

[0065] Please continue to combine Figure 4 As shown, if the center pixel A0 has pixels A1, A2, and A3 above it and pixels A6, A7, and A8 below it, then:

[0066] ;

[0067] C2 is the difference between the pixels above and below the center pixel A0. The three-channel color values ​​of the pixel above or below the center pixel A0.

[0068] The calculated C2 value is compared with the second threshold. When it is greater than the second threshold, the center pixel is determined as the second feature point. Then: a coordinate system is constructed with one of the corners of the feature map of the image to be matched as the origin of the coordinate system. Each pixel is used as a coordinate system unit value. Each second feature point is mapped to a point in the coordinate system. For each point in the coordinate system, when there is a second feature point at the point, the value is taken as the difference C2 corresponding to the second feature point. When there is no second feature point, the value is taken as 0, thus obtaining the second feature matrix to be matched.

[0069] The method for determining the third feature matrix of the image feature map to be matched is as follows: For the extracted pixel block, calculate the difference between the left and right pixels of the center pixel of each pixel block, determine the third feature point from the center pixel of each pixel block based on the difference, and map the third feature matrix based on the position of the third feature point.

[0070] Please continue to combine Figure 4 As shown, the center pixel A0 has pixels A1, A4, and A6 to its left and pixels A3, A5, and A8 to its right. Therefore:

[0071] ;

[0072] C3 is the difference between the pixels to the left and right of the center pixel A0. ), ( ), ( ), ( ), ( ), ( These are the three-channel color values ​​of the left-hand pixels A1, A4, and A6, and the right-hand pixels A3, A5, and A8 of the center pixel A0.

[0073] The calculated C3 value is compared with the third threshold. When it is greater than the third threshold, the center pixel is determined as the third feature point. Then, a coordinate system is constructed with one corner of the feature map of the image to be matched as the origin of the coordinate system. Each pixel is used as a coordinate system unit value. Each third feature point is mapped to a point in the coordinate system. For each point in the coordinate system, when there is a third feature point at that point, the value is taken as the difference C3 corresponding to that third feature point. When there is no third feature point, the value is taken as 0, thus obtaining the third feature matrix to be matched.

[0074] Step S3: Obtain the second edge matrix of the target-size image to be identified, and filter the initial images to be matched from the images to be matched based on the edge similarity between the second edge matrix and the first edge matrix.

[0075] The method for obtaining the second edge matrix of the target-size image to be identified is similar to that for obtaining the first edge matrix: First, the edges of the target-size image to be identified are obtained through an edge detection algorithm. Then, a coordinate system is constructed with one corner of the target-size image to be identified as the origin of the coordinate system. Each pixel is used as a unit value of the coordinate system, and each pixel of the image edge is mapped to a point in the coordinate system. For each point in the coordinate system, the value is 1 when there is an edge pixel at the point, and the value is 0 when there is no edge pixel at the point, thus obtaining the second edge matrix.

[0076] When acquiring the second edge matrix, the origin of the coordinate system of the target-size image to be identified is consistent with the origin of the coordinate system of the target-size image to be matched when acquiring the first edge matrix, so as to ensure that the target-size image to be identified and the target-size image to be matched have the same comparison benchmark.

[0077] Step S4: Obtain the feature map of the target-size image to be identified, as well as the first feature matrix, the second feature matrix, and the third feature matrix of the feature map to be identified.

[0078] Similar to the feature map of the image to be matched, the feature map of the image to be identified is obtained by multiple rounds of feature map generation units performing feature extraction and resolution enhancement on the target-size image to be identified. Each round of feature map generation unit includes a first branch and a second branch. The first branch performs channel feature normalization, nonlinear transformation, convolutional layer feature extraction, and size enlargement on the input image. The second branch performs resolution enhancement on the input image. Then, the feature maps processed by the first and second branches are fused, and the fused feature map is output to the next feature map generation unit. The feature map output by the last round of feature map generation unit is the feature map of the image to be identified.

[0079] The method for obtaining the first feature matrix to be identified of the feature map to be identified is similar to that for obtaining the first feature matrix to be matched of the feature map to be matched: the feature map to be identified is divided into several pixel blocks, several pixel blocks are extracted from them, for each extracted pixel block, the difference between the center pixel of each pixel block and other pixels in the pixel block is calculated, the first feature point is determined from the center pixel of each pixel block based on the difference, and the first feature matrix to be identified is mapped based on the position of the first feature point.

[0080] The method for obtaining the second feature matrix to be identified of the feature map to be identified is similar to that for obtaining the second feature matrix to be matched of the feature map to be matched: the feature map to be identified is divided into several pixel blocks, and several pixel blocks are extracted from them. For each extracted pixel block, the difference between the pixel above and the pixel below the center pixel of each pixel block is calculated. Based on the difference, the second feature point is determined from the center pixel of each pixel block. The second feature matrix to be identified is mapped based on the position of the second feature point.

[0081] The method for obtaining the third feature matrix to be identified in the feature map of the image to be identified is similar to that for obtaining the third feature matrix to be matched in the feature map of the image to be matched: the feature map of the image to be identified is divided into several pixel blocks, and several pixel blocks are extracted from them. For each extracted pixel block, the difference between the left and right pixels of the center pixel of each pixel block is calculated. Based on the difference, the third feature point is determined from the center pixel of each pixel block, and the third feature matrix is ​​mapped based on the position of the third feature point.

[0082] It should be noted that when mapping the first feature matrix to be identified and the first feature matrix to be matched, the origin of the coordinate system of the feature map to be identified and the feature map to be matched should be consistent, and the pixel block division method and the position of the extracted pixel blocks should also be completely consistent; when mapping the second feature matrix to be identified and the second feature matrix to be matched, the origin of the coordinate system of the feature map to be identified and the feature map to be matched should also be consistent, and the pixel block division method and the position of the extracted pixel blocks should also be completely consistent; when mapping the third feature matrix to be identified and the third feature matrix to be matched, the origin of the coordinate system of the feature map to be identified and the feature map to be matched should also be consistent, and the pixel block division method and the position of the extracted pixel blocks should also be completely consistent.

[0083] When mapping the first, second, and third feature matrices to be identified, or mapping the first, second, and third feature matrices to be matched, the pixel block division method of the feature map to be identified or the feature map to be matched, as well as the position of the extracted pixel blocks, can be the same or different. When they are different, the first / second feature matrices to be identified / matched, the second / third feature matrices to be identified / matched, and the third feature matrices to be matched can be obtained based on different pixel block layouts. The final similarity can reflect the similarity between images from different pixel block layouts, making the similarity comparison more comprehensive and authoritative.

[0084] Step S5: For each initially selected image to be identified, obtain the first, second, and third feature matrices of the corresponding image feature map from the image database. Determine a first similarity based on the first and second feature matrices; determine a second similarity based on the second and third feature matrices; determine a third similarity based on the third feature matrix; and determine a feature similarity based on the first, second, and third similarities. This feature similarity is the feature similarity between the image feature map to be identified and the initial image feature map to be matched, and also the feature similarity between the image to be identified and the initial image to be matched. Based on the feature similarity between the initial image to be matched and the image to be identified, determine the matching image that matches the image to be identified, as the recognition result.

[0085] Feature similarity can be the average of the first, second, and third similarities, or it can be a weighted sum of the first, second, and third similarities.

[0086] Therefore, the core technical concept and technical effects of the multimodal target image recognition method of the present invention are as follows:

[0087] 1. The image is coarsely screened based on the edge detection algorithm. At this time, the similarity of the edge pixel positions of the original image is calculated. Only the initial images that meet the similarity are subjected to further in-depth calculations, which greatly reduces the workload of subsequent calculations in the initial stage.

[0088] 2. By performing detailed feature extraction and resolution enhancement on the original image through a multi-round feature map generation unit, it is helpful to mine deep features during subsequent feature matching. This facilitates the appropriate reduction of the number of subsequent pixel blocks without affecting the feature matching effect, thus further reducing the matrix dimension of the feature matching process, further reducing the amount of computation, reducing the recognition time, and improving the recognition efficiency.

[0089] 3. Considering that feature points involve color value transitions, and for existing images involving different types and characteristics of color value transitions, three feature matrices were introduced, and three different color value difference calculation methods were matched. Taking into account the color value transitions in different directions of the image, the extraction of detailed features from multiple directions of the image was realized, ensuring the accuracy of the final recognition result.

[0090] Although the present invention has been described using the above preferred embodiments, it is not intended to limit the scope of protection of the present invention. Any changes and modifications made by those skilled in the art to the above embodiments without departing from the spirit and scope of the present invention shall still fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be defined by the claims.

Claims

1. A target image recognition method based on multimodal technology, characterized in that... include: Step S1: Obtain the image to be recognized as the target image, and preprocess it to obtain the target-sized image to be recognized; Step S2: Construct a database of images to be matched. The database stores the following for each image to be matched: the target size image to be matched, the first edge matrix of the target size image to be matched, and the feature map of the image to be matched. Step S3: Obtain the second edge matrix of the target-size image to be identified, and filter the initial images to be matched from the images to be matched based on the edge similarity between the second edge matrix and the first edge matrix; Step S4: Obtain the feature map of the image to be identified of the target size image to be identified. Based on the feature similarity between the feature map of the image to be identified and the feature map of the initial image to be matched, determine the matching image that matches the image to be identified, and use it as the recognition result. The database of images to be matched also stores the first feature matrix, the second feature matrix, and the third feature matrix of the feature maps of the images to be matched; Methods for obtaining the feature similarity between the feature map of the image to be identified and the feature map of the initial image to be matched include: Determine the first, second, and third feature matrices of the feature map to be identified in the image to be identified; A first similarity is determined based on the first feature matrix to be identified and the first feature matrix to be matched in the initial image feature map; a second similarity is determined based on the second feature matrix to be identified and the second feature matrix to be matched in the initial image feature map; a third similarity is determined based on the third feature matrix to be identified and the third feature matrix to be matched in the initial image feature map. Feature similarity is determined based on the first similarity, second similarity, and third similarity. The method for determining the first feature matrix of the image feature map to be matched includes: dividing the image feature map to be matched into several pixel blocks, determining the first feature point A based on the difference between the center pixel of the pixel block and other pixels in the pixel block, and mapping the first feature matrix to be matched based on the position of the first feature point A. The method for determining the first feature matrix of the image feature map to be identified includes: dividing the image feature map to be identified into several pixel blocks, determining the first feature point B based on the difference between the center pixel of the pixel block and other pixels in the pixel block, and mapping the first feature matrix to be identified based on the position of the first feature point B. The method for determining the second feature matrix of the image feature map to be matched includes: dividing the image feature map to be matched into several pixel blocks, determining the second feature point A based on the difference between the pixel above and the pixel below the center point of the pixel block, and mapping the second feature matrix based on the position of the second feature point A; The method for determining the second feature matrix of the image feature map to be identified includes: dividing the image feature map to be identified into several pixel blocks, determining the second feature point B based on the difference between the pixel above and the pixel below the center point of the pixel block, and mapping the second feature matrix based on the position of the second feature point B. The method for determining the third feature matrix of the feature map of the image to be matched includes: dividing the feature map of the image to be matched into several pixel blocks, determining the third feature point A based on the difference between the left and right pixels of the center pixel of the pixel block, and mapping the third feature matrix based on the position of the third feature point A. The method for determining the third feature matrix of the image feature map to be identified includes: dividing the image feature map to be identified into several pixel blocks, determining the third feature point B based on the difference between the left and right pixels of the center pixel of the pixel block, and mapping the third feature matrix based on the position of the third feature point B.

2. The target image recognition method based on multimodal technology as described in claim 1, characterized in that: The target size image to be matched is obtained by scaling the image to be matched proportionally and filling it with black pixels; The target size image to be identified is obtained by scaling the image to be identified proportionally and filling it with black pixels.

3. The target image recognition method based on multimodal technology as described in claim 1, characterized in that: The method for obtaining the first edge matrix of the target-size image to be matched includes: obtaining the edges of the target-size image to be matched through an edge detection algorithm, and mapping the edges of the target-size image to be matched into the first edge matrix based on the position of the pixel points of the image edges; The method for obtaining the second edge matrix of the target-size image to be identified includes: obtaining the edges of the target-size image to be identified through an edge detection algorithm, and mapping the edges of the target-size image to be identified into a second edge matrix based on the position of the pixel points of the image edges.

Images