Image matching method and device for non-equivalent pixels, computer device and storage medium

Abstract

The application discloses an image matching method and device of unequal pixels, computer equipment and a storage medium. The method comprises the following steps: dividing at least two images to be matched into the same number of image blocks; inputting the divided at least two images to be matched into a trained feature extraction model, and extracting the overall feature vector of the at least two images to be matched and the feature vector of each image block by using the feature extraction model; calculating the overall matching degree value of the at least two images to be matched according to the overall feature vector of the at least two images to be matched, and calculating the matching degree value of the image blocks corresponding in position in the at least two images to be matched according to the feature vector of the image blocks; and merging the overall matching degree value of the images to be matched and the matching degree value of the image blocks corresponding in position to obtain the matching degree calculation result of the at least two images to be matched. The application adds block matching on the basis of overall image matching degree comparison, thereby greatly improving the image similarity calculation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image matching, in particular to an image matching method and device for non-equivalent pixels, a computer device and a storage medium. BACKGROUND

[0002] Image recognition is a technology that uses computers to process, analyze and understand images to identify various different patterns of targets and objects. Image recognition is one of the important technologies in the field of AI (Artificial Intelligence) in recent years, such as face recognition, living body detection, license plate recognition, illegal image positioning and other technologies based on image recognition. The image matching algorithm is also an important branch. The image matching algorithm extracts features from the input two images, calculates the difference between the feature values, obtains the matching degree of the two images, and finally finds the similar image of the input image.

[0003] At present, the mainstream image matching degree calculation algorithm includes Hamming distance, Euclidean distance, perceptual hashing, etc. Although the existing algorithm can meet the actual application requirements to a certain extent, the comparison effect of images with different pixels or different depth of field is not satisfactory. At the same time, due to the change of feature values after resizing of images with different pixels, the result of unilateral horizontal calculation of features is slightly rough, resulting in inaccurate matching degree calculation result. SUMMARY

[0004] The present application provides an image matching method and device for non-equivalent pixels, a computer device and a storage medium, aiming to solve the technical problems of existing image matching degree calculation algorithm for images with different pixels or different depth of field, and the matching effect is poor, and the matching degree calculation result is not accurate enough.

[0005] To solve the above technical problems, the technical scheme adopted by the present application is as follows:

[0006] An image matching method for non-equivalent pixels comprises:

[0007] Divide at least two images to be matched into the same number of image blocks according to a set segmentation ratio;

[0008] Input the segmented at least two images to be matched into a trained feature extraction model, and use the feature extraction model to extract the overall feature vector of the at least two images to be matched and the feature vector of each image block;

[0009] Calculate the overall matching degree value of the at least two images to be matched according to the overall feature vector of the at least two images to be matched, and calculate the matching degree value of the image blocks corresponding in position in the at least two images to be matched according to the feature vector of the image blocks.

[0010] The overall matching degree value of the to-be-matched image and the matching degree value of the image block corresponding to the position are combined to obtain a matching degree calculation result of the at least two to-be-matched images.

[0011] The technical scheme adopted by the embodiment of the application further includes that the at least two to-be-matched images are divided into the same number of image blocks according to the set segmentation ratio, and the dividing further includes:

[0012] The image blocks in the at least two to-be-matched images are marked according to a set marking order;

[0013] The image blocks corresponding to the positions in the at least two to-be-matched images are combined as block groups according to the marking order of the image blocks.

[0014] The technical scheme adopted by the embodiment of the application further includes that the feature extraction model is a VGG model, the VGG model includes a transforms module, a convolutional layer, a pooling layer, a Flatten layer, and a fully connected layer, and the extracting of the overall feature vector of the at least two to-be-matched images and the feature vector of each image block by using the feature extraction model specifically includes:

[0015] The to-be-matched image is input into the transforms module after being encoded, and the output of the transforms module is output after being processed by multi-head attention sequence, the output of the transforms module is input into the convolutional layer and the pooling layer for feature vector extraction, the feature vectors output by the convolutional layer and the pooling layer are input into the Flatten layer for dimension reduction processing, and finally the feature vector extraction result is output through the fully connected layer.

[0016] The technical scheme adopted by the embodiment of the application further includes that the overall matching degree value of the at least two to-be-matched images is calculated according to the overall feature vector of the at least two to-be-matched images, and the matching degree value of the image block corresponding to the region in the at least two to-be-matched images is calculated according to the feature vector of the image block, and the calculating further includes:

[0017] The overall feature vector of the at least two to-be-matched images and the feature vector of the corresponding block combination are compared to obtain the overall matching degree value of the at least two to-be-matched images and the matching degree value of the corresponding block combination, respectively.

[0018] According to a set screening rule, the block combination with a matching degree value lower than a set threshold value is deleted to obtain a screened block combination.

[0019] The technical scheme adopted by the embodiment of the application further includes that the combining of the overall matching degree value of the to-be-matched image and the matching degree value of the corresponding image block specifically includes:

[0020] The overall matching degree value of the at least two images to be matched and the matching degree value of the screened block combination are combined.

[0021] The technical scheme adopted by the embodiment of the application further includes that the matching degree calculation of the at least two images to be matched is specifically:

[0022]

[0023] Supposing that the number of the screened block combinations is four, the above formula is expanded as:

[0024] t1*f1(x1,y1)+t2*f2(x2,y2)+t3*f3(x3,y3)+t4*f4(x4,y4)+t0*f0(x0,y0)+bias

[0025] Wherein, t1, t2, t3 and t4 are contrast weights of the four block combinations respectively, and t0 represents a contrast weight of the image to be matched. i n is a weight value before normalization, i represents different image blocks, n is the number of the images to be matched and the block combinations; f(x, y) is a cosine calculation function, f(x, y) = cosine_similarity(x, y), x and y are feature vectors of corresponding block combinations respectively, and bias is a bias term.

[0026] Another technical scheme adopted by the embodiment of the application is an image matching device for non-equivalent pixels, which includes:

[0027] An image segmentation module is configured to segment at least two images to be matched into image blocks in the same number according to a set segmentation ratio;

[0028] A feature extraction module is configured to input the segmented at least two images to be matched into a trained feature extraction model, and extract overall feature vectors of the at least two images to be matched and feature vectors of each image block by using the feature extraction model;

[0029] A first matching value calculation module is configured to calculate an overall matching degree value of the at least two images to be matched according to the overall feature vectors of the at least two images to be matched, and calculate matching degree values of image blocks corresponding in position in the at least two images to be matched according to the feature vectors of the image blocks;

[0030] A second matching value calculation module is configured to combine the overall matching degree value of the at least two images to be matched and the matching degree values of the image blocks corresponding in position, to obtain a matching degree calculation result of the at least two images to be matched.

[0031] The technical scheme adopted by the embodiment of the present application further includes: the first matching value calculation module calculates an overall matching degree value of the at least two images to be matched according to the overall feature vectors of the at least two images to be matched, and calculates a matching degree value of the image blocks corresponding in position in the at least two images to be matched according to the feature vectors of the image blocks, specifically:

[0032] The image blocks in the at least two images to be matched are marked according to a set marking order, and the image blocks corresponding in position in the at least two images to be matched are combined as block groups according to the marking order of the image blocks; the overall feature vectors of the at least two images to be matched and the feature vectors of the corresponding block groups are compared, and the overall matching degree value of the at least two images to be matched and the matching degree value of the corresponding block groups are obtained respectively.

[0033] Still another technical scheme adopted by the embodiment of the present application is a computer device, which comprises:

[0034] a memory storing executable program codes;

[0035] a processor connected with the memory;

[0036] The processor invokes the executable program codes stored in the memory to execute the image matching method of non-equivalent pixels as described above.

[0037] Still another technical scheme adopted by the embodiment of the present application is a storage medium storing program instructions executable by a processor, and the program instructions are used to execute the image matching method of non-equivalent pixels as described above.

[0038] The image matching method of non-equivalent pixels, the device, the computer device and the storage medium of the embodiment of the present application calculate the overall matching degree value of the images to be matched and the matching degree value of the corresponding image blocks according to the overall feature vectors and the feature vectors of the image blocks respectively by dividing the images to be matched and extracting the feature vectors of each image block at the same time, and then the overall matching degree value and the matching degree value of the corresponding image blocks are combined after the image blocks with lower matching degree values are deleted, to obtain the matching degree calculation result of the two images to be matched. The present application adds block matching on the basis of overall image matching degree comparison, greatly improves the image matching degree calculation result, effectively avoids the simple and rough calculation mode of single feature calculation through the mode of regional comparison and multi-region linear combination, thereby greatly improving the image similarity calculation result under non-equivalent pixels and sizes, and making the similar image matching value in a more reasonable range. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a flowchart of the image matching method of non-equivalent pixels of the first embodiment of the present application;

[0040] Figure 2 is a flowchart of the image matching method of the unequal pixels of the second embodiment of the present application;

[0041] Figure 3 is a schematic diagram of the image segmentation result of the embodiment of the present application;

[0042] Figure 4 is a schematic diagram of the image feature extraction process of the embodiment of the present application;

[0043] Figure 5 is a structural schematic diagram of the image matching device of the unequal pixels of the embodiment of the present application;

[0044] Figure 6 is a structural schematic diagram of the computer device of the embodiment of the present application;

[0045] Figure 7 is a structural schematic diagram of the storage medium of the embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0047] The terms "first", "second", "third" in the present application are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second", "third" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise specifically limited. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, movement condition, etc. between the components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications also change accordingly. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, the process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0048] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described herein can be combinable with other embodiments.

[0049] Reference is made to Figure 1 is a flowchart of the image matching method of the non-equivalent pixels of the first embodiment of the application. The image matching method of the non-equivalent pixels of the first embodiment of the application comprises the following steps:

[0050] S100: dividing at least two images to be matched into the same number of image blocks according to a set segmentation ratio;

[0051] S110: inputting the segmented at least two images to be matched into a trained feature extraction model, and extracting the overall feature vector of the at least two images to be matched and the feature vector of each image block by using the feature extraction model;

[0052] S120: calculating the overall matching degree value of the at least two images to be matched according to the overall feature vector of the at least two images to be matched, and calculating the matching degree value of the image blocks corresponding in position in the at least two images to be matched according to the feature vector of the image blocks;

[0053] S130: merging the overall matching degree value of the images to be matched and the matching degree value of the image blocks corresponding in position to obtain the matching degree calculation result of the at least two images to be matched.

[0054] Based on the above, the image matching method of the non-equivalent pixels of the first embodiment of the application divides the images to be matched into blocks, extracts the overall feature vector of the images and the feature vector of each image block at the same time, calculates the overall matching degree value of the images to be matched and the matching degree value of the corresponding image blocks according to the overall feature vector and the feature vector of the image blocks, and merges the overall matching degree value and the matching degree value of the corresponding image blocks to obtain the matching degree calculation result of the two images to be matched. The application adds block matching on the basis of overall image matching degree comparison, thereby greatly improving the image similarity calculation result under non-equivalent pixels and sizes, and making the similarity image matching value within a more reasonable range.

[0055] Reference is made to Figure 2 is a flowchart of the image matching method of the non-equivalent pixels of the second embodiment of the application. The image matching method of the non-equivalent pixels of the second embodiment of the application comprises the following steps:

[0056] S200: obtaining two images to be matched with non-equivalent pixels / size;

[0057] In this step, the unequal pixels refer to images with different depth of field (DOF, which refers to the distance range in front of and behind the object measured by the imaging of the clear image obtained in front of the camera lens or other imager).

[0058] S210: The two images to be matched are divided into blocks according to the set division ratio, and the same number of image blocks of the two images to be matched are generated, each image block is labeled according to the set order, and the image blocks corresponding in position in the two images to be matched are combined as a block group according to the label order of the image blocks.

[0059] In this step, please refer to Figure 3 is a schematic diagram of the image segmentation result of the embodiment of the application. Assuming that the two images to be matched are image A and image B, image A and image B are divided into four image blocks according to the average division method, and the four image blocks of image A are labeled as a0, a1, a2, and a3 in the label order from left to right and from top to bottom, and the four image blocks of image B are labeled as c0, c1, c2, and c3. The image blocks after division are labeled to facilitate the subsequent calculation of the matching degree of the image blocks. It can be understood that the division ratio and the number of divisions of the image can be set according to the actual application scenario, and the division ratio is not limited to average division, and the division ratio of the image blocks in the same region of the two images to be matched can also be set respectively, as long as the division ratio of the image blocks in the same region of the two images to be matched is the same (i.e., the division ratio of a0 and c0, a1 and c1, a2 and c2, and a3 and c3 is the same), which belongs to the protection scope of the embodiment of the application. Taking Figure 3 for example, the image blocks corresponding in position in the two images to be matched are combined as a block group according to the label order of the image blocks, which is specifically: a0 and c0, a1 and c1, a2 and c2, and a3 and c3 are combined as a block group.

[0060] S220: The two images to be matched after division are input into the trained feature extraction model, and the feature extraction model is used to extract the overall feature vector of the two images to be matched and the feature vector of each image block.

[0061] In this step, specifically as Figure 4As shown, it is a schematic diagram of the image feature extraction process of the embodiment of the present application. The feature extraction process is as follows: the input image enters the transforms module for multi-head attention sequence processing after being encoded, and outputs the feature vector of the image; then the feature vector enters the convolution layer and the pooling layer in turn, and the non-linear expression ability of the feature vector is increased through the convolution layer and the pooling layer. The output of the convolution layer and the pooling layer is processed by the Flatten layer for dimension reduction, which converts the multi-dimensional input into one-dimensional, and finally the feature vector extraction result is output through the fully connected layer. In the embodiment of the present application, in order to facilitate calculation, the output feature vectors are marked as sa0, sa1, sa2, sa3, sa and sc0, sc1, sc2, sc3 respectively, wherein sa0, sa1, sa2, sa3 and sc0, sc1, sc2, sc3 are the feature vectors of image blocks a0, a1, a2, a3 and c0, c1, c2, c3 respectively, and sa and sc are the overall feature vectors of image A and image B respectively.

[0062] Further, the feature extraction model in the embodiment of the present application is a VGG model under pytorch. The model relies on tensorflow, and the corresponding weight value i n and the offset term bias are trained by comparing the original pixels and sizes of the two input images. The training samples of the model are a plurality of image combinations with different pixels and sizes, and the matching degree range of the image combination is manually labeled. The model training process includes:

[0063] Step 1, construct a multiple linear model: SIM = f0*t0 + … + f4*t4 + bias; wherein, f i represents a basic similarity calculation function, i represents different image blocks, t n represents the weight of different image blocks, and n represents the number of images (blocks); wherein the multiple linear model includes 3 convolution layers + 2 fully connected layers, and the regression equation used in the model can be designed according to the business scenario.

[0064] Step 2, use a quadratic cost function to calculate the difference between the predicted weight and offset term of the model and the manually labeled data;

[0065] Step 3, define the gradient descent algorithm (train.GradientDescentOptimizer), activate the neural network to start iterative training, and make the model quickly iterate to the fitting state.

[0066] S230: Compare the overall feature vectors of the two images to be matched and the feature vectors of the corresponding block combinations, and obtain the overall matching degree value of the two images to be matched and the matching degree value of the corresponding block combinations respectively;

[0067] In this step, the obtained matching degree values are s0, s1, s2, s3, and s, where s0, s1, s2, and s3 are Figure 3 The matching degree values of a0 and c0, a1 and c1, a2 and c2, and a3 and c3 are s0, s1, s2, and s3, respectively, and s is the overall matching degree value of the image A and the image B.

[0068] S240: According to the set screening rule, the block combination with a matching degree value lower than the set threshold value is deleted to obtain a screened block combination.

[0069] In this step, the threshold value of the screened image block is set to 50%, that is, if the matching degree value of two image blocks is lower than 50%, the two image blocks are deleted. The specific threshold setting range can be adjusted according to the actual application scenario.

[0070] S250: The overall matching degree value of the two images to be matched and the matching degree value of the screened block combination are combined to obtain the final matching degree calculation result of the two images to be matched.

[0071] In this step, the matching degree calculation formula of the two images to be matched is specifically:

[0072]

[0073] As follows:

[0074] t1*f1(x1,y1)+t2*f2(x2,y2)+t3*f3(x3,y3)+t4*f4(x4,y4)+t0*f0(x0,y0)+bias (2)

[0075] Where t1, t2, t3, and t4 are the contrast weights of the four block combinations, and t0 represents the contrast weight of the image to be matched. in is the weight value before normalization, i represents different image blocks, n is the number of images (blocks), f(x, y) is a cosine calculation function, f(x, y) = cosine_similarity(x, y), x and y are feature vectors of corresponding image blocks, and bias is a bias term.

[0076] Based on the above, the image matching method of the second embodiment of the present application for non-equivalent pixels extracts the feature vector of each image block while extracting the overall feature vector of the image by dividing the image to be matched, calculates the overall matching degree value of the image to be matched and the matching degree value of the corresponding image block according to the overall feature vector and the feature vector of the image block, deletes the image block with a lower matching degree value, and combines the overall matching degree value and the matching degree value of the corresponding image block to obtain the matching degree calculation result of the two images to be matched. The present application adds block matching to the overall image matching degree comparison, greatly improves the image matching degree calculation result, effectively avoids the simple and rough calculation method of single feature calculation through regional comparison and multi-region linear combination, thereby greatly improving the image similarity calculation result under non-equivalent pixels and sizes, and making the similar image matching value within a more reasonable range.

[0077] In an optional embodiment, the result of the image matching method for non-equivalent pixels can also be uploaded to a blockchain.

[0078] Specifically, the corresponding abstract information is obtained based on the result of the image matching method for non-equivalent pixels, and specifically, the abstract information is obtained by hash processing of the result of the image matching method for non-equivalent pixels, for example, by processing using the sha256s algorithm. Uploading the abstract information to the blockchain can ensure its security and fair transparency to users. Users can download the abstract information from the blockchain to verify whether the result of the image matching method for non-equivalent pixels has been tampered with. The blockchain referred to in this example is a new application mode of computer technology such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. Blockchain, in essence, is a decentralized database, a series of data blocks associated using cryptographic methods, each containing information about a batch of network transactions, used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0079] Please refer to Figure 5 is a structural schematic diagram of the image matching device for non-equivalent pixels according to the embodiment of the present application. The image matching device 40 for non-equivalent pixels according to the embodiment of the present application comprises:

[0080] The image segmentation module 41 is configured to segment at least two images to be matched into the same number of image blocks according to a set segmentation ratio;

[0081] The feature extraction module 42 is configured to input the segmented at least two images to be matched into a trained feature extraction model, and extract the overall feature vector of the at least two images to be matched and the feature vector of each image block using the feature extraction model;

[0082] The first matching value calculation module 43 is configured to calculate the overall matching degree value of the at least two images to be matched according to the overall feature vectors of the at least two images to be matched, and calculate the matching degree value of the image blocks corresponding in position in the at least two images to be matched according to the feature vectors of the image blocks.

[0083] The second matching value calculation module 44 is configured to merge the overall matching degree value of the images to be matched and the matching degree value of the image blocks corresponding in position to obtain the matching degree calculation result of the at least two images to be matched.

[0084] The image matching device for non-equivalent pixels in the embodiment of the application can obtain the matching degree calculation result of the at least two images to be matched by dividing the images to be matched, extracting the feature vectors of the overall images and the feature vectors of each image block at the same time, calculating the overall matching degree value of the images to be matched and the matching degree value of the corresponding image blocks according to the overall feature vectors and the feature vectors of the image blocks, and merging the overall matching degree value and the matching degree value of the corresponding image blocks. The image similarity calculation result under the non-equivalent pixels and the size is greatly improved on the basis of the overall image matching degree comparison, so that the matching value of the similar images is in a more reasonable range.

[0085] Please refer to Figure 6 The computer device 50 includes:

[0086] The memory 51 stores executable program codes;

[0087] The processor 52 is connected with the memory 51;

[0088] The processor 52 is configured to call the executable program codes stored in the memory 51 and perform the following steps: dividing the at least two images to be matched into the same number of image blocks according to a set segmentation ratio; inputting the segmented at least two images to be matched into a trained feature extraction model, and extracting the overall feature vectors of the at least two images to be matched and the feature vectors of each image block by using the feature extraction model; calculating the overall matching degree value of the at least two images to be matched according to the overall feature vectors of the at least two images to be matched, and calculating the matching degree value of the image blocks corresponding in position in the at least two images to be matched according to the feature vectors of the image blocks; merging the overall matching degree value of the images to be matched and the matching degree value of the image blocks corresponding in position to obtain the matching degree calculation result of the at least two images to be matched.

[0089] The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 can be an integrated circuit chip having a processing capability of signals. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like.

[0090] The computer device of the embodiment of the present application divides the to-be-matched images, extracts the feature vectors of the whole images and the feature vectors of each image block at the same time, calculates the whole matching degree value of the to-be-matched images and the matching degree value of the corresponding image blocks according to the whole feature vectors and the feature vectors of the image blocks, merges the whole matching degree value and the matching degree value of the corresponding image blocks, and obtains the matching degree calculation result of the two to-be-matched images. The present application adds the block matching on the basis of the whole image matching degree comparison, thereby greatly improving the image similarity calculation result under the condition of the non-equivalent pixels and sizes, and making the matching value of the similar images in a more reasonable range.

[0091] Please refer to Figure 7 , Figure 7A structural schematic diagram of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores a program file 61 capable of implementing the following steps: dividing at least two to-be-matched images into the same number of image blocks according to a set segmentation ratio; inputting the segmented at least two to-be-matched images into a trained feature extraction model, and extracting an overall feature vector of the at least two to-be-matched images and a feature vector of each image block by using the feature extraction model; calculating an overall matching degree value of the at least two to-be-matched images according to the overall feature vector of the at least two to-be-matched images, and calculating a matching degree value of a position corresponding image block in the at least two to-be-matched images according to the feature vector of the image block; and merging the overall matching degree value of the to-be-matched images and the matching degree value of the position corresponding image block to obtain a matching degree calculation result of the at least two to-be-matched images. The program file 61 can be stored in the above storage medium in the form of a software product, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes, or a terminal device such as a computer, a server, a mobile phone, or a tablet.

[0092] The storage medium of the embodiment of the present application divides the to-be-matched images into blocks, extracts an overall feature vector of the images and a feature vector of each image block at the same time, calculates an overall matching degree value of the to-be-matched images and a matching degree value of the corresponding image block according to the overall feature vector and the feature vector of the image block, merges the overall matching degree value and the matching degree value of the corresponding image block, and obtains a matching degree calculation result of the two to-be-matched images. The present application adds block matching on the basis of overall image matching degree comparison, thereby greatly improving the image similarity calculation result under the condition of unequal pixels and sizes, and making the similarity image matching value in a more reasonable range.

[0093] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the division of the system embodiments is only a logical function division, and there can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0094] In addition, each function unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software function unit. The above is only an implementation manner of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied to other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method of image matching of non-equivalent pixels, characterized in that, The method comprises the following steps: segmenting at least two to-be-matched images into the same number of image blocks according to a set segmentation ratio; inputting the segmented at least two to-be-matched images into a trained feature extraction model to extract an overall feature vector of the at least two to-be-matched images and a feature vector of each image block by using the feature extraction model; calculating an overall matching degree value of the at least two to-be-matched images according to the overall feature vector of the at least two to-be-matched images, and calculating a matching degree value of the image blocks corresponding in position in the at least two to-be-matched images according to the feature vector of the image blocks; merging the overall matching degree value of the to-be-matched images and the matching degree value of the corresponding image blocks to obtain a matching degree calculation result of the at least two to-be-matched images; segmenting at least two to-be-matched images into the same number of image blocks according to a set segmentation ratio further comprises: labeling the image blocks in the at least two to-be-matched images according to a set labeling order; combining the image blocks corresponding in position in the at least two to-be-matched images as block groups according to the labeling order of the image blocks; according to a set screening rule, deleting the block groups with a matching degree value lower than a set threshold value to obtain screened block groups.

2. The image matching method of unequal pixels according to claim 1, characterized in that, The feature extraction model is a VGG model, the VGG model comprises a transforms module, a convolutional layer, a pooling layer, a Flatten layer and a fully connected layer, and the extraction of the overall feature vector of the at least two to-be-matched images and the feature vector of each image block by using the feature extraction model specifically comprises: inputting the to-be-matched images into the transforms module for multi-head attention sequence processing, outputting the transforms module, inputting the output of the transforms module into the convolutional layer and the pooling layer for feature vector extraction, inputting the feature vectors output by the convolutional layer and the pooling layer into the Flatten layer for dimension reduction processing, and finally outputting the feature vector extraction result through the fully connected layer.

3. The method of claim 1, wherein, The calculation of the overall matching degree value of the at least two to-be-matched images according to the overall feature vector of the at least two to-be-matched images, and the calculation of the matching degree value of the image blocks corresponding in region in the at least two to-be-matched images according to the feature vector of the image blocks comprises: comparing the overall feature vector of the at least two to-be-matched images and the feature vector of the corresponding block groups to obtain the overall matching degree value of the at least two to-be-matched images and the matching degree value of the corresponding block groups, respectively; according to a set screening rule, deleting the block groups with a matching degree value lower than a set threshold value to obtain screened block groups.

4. The image matching method of unequal pixels according to claim 3, characterized in that, The merging of the overall matching degree value of the to-be-matched images and the matching degree value of the corresponding image blocks specifically comprises: merging the overall matching degree value of the at least two to-be-matched images and the matching degree value of the screened block groups.

5. The method of image matching of non-equal pixels according to claim 4, wherein, The matching degree calculation of the at least two to-be-matched images specifically comprises: assuming that the number of the screened block groups is 4, the above formula is expanded as: t1*f1(x1,y1)+t2*f2(x2,y2)+t3*f3(x3,y3)+t4*f4(x4,y4)+t0*f0(x0,y0)+bias Wherein, t1, t2, t3, t4 are contrast weights of four block combinations respectively, and t0 represents the contrast weight of the image to be matched. i n is the normalized weight value, i represents different image blocks, n is the number of images to be matched and block combinations; f(x, y) is a cosine calculation function, f(x, y) = cosine_similarity(x, y), x and y are feature vectors of corresponding block combinations, and bias is a bias term.

6. An apparatus for mismatched pixel image matching, the apparatus being for implementing the mismatched pixel image matching method according to any one of claims 1 to 5, characterized in that, comprising: an image segmentation module, configured to segment at least two images to be matched into the same number of image blocks according to a set segmentation ratio; a feature extraction module, configured to input the segmented at least two images to be matched into a trained feature extraction model, and extract an overall feature vector of the at least two images to be matched and a feature vector of each image block by using the feature extraction model; a first matching value calculation module, configured to calculate an overall matching degree value of the at least two images to be matched according to the overall feature vector of the at least two images to be matched, and calculate a matching degree value of image blocks corresponding in position in the at least two images to be matched according to the feature vector of the image blocks; a second matching value calculation module, configured to combine the overall matching degree value of the images to be matched and the matching degree value of the image blocks corresponding in position to obtain a matching degree calculation result of the at least two images to be matched.

7. The apparatus for image matching of unequal pixels of claim 6, wherein, The first matching value calculation module calculates an overall matching degree value of the at least two images to be matched according to the overall feature vector of the at least two images to be matched, and calculates a matching degree value of image blocks corresponding in position in the at least two images to be matched according to the feature vector of the image blocks, specifically as follows: The image blocks in the at least two images to be matched are marked according to a set marking order, and image blocks corresponding in position in the at least two images to be matched are combined as block groups according to the marking order of the image blocks; the overall feature vector of the at least two images to be matched and the feature vector of the corresponding block groups are compared to obtain an overall matching degree value of the at least two images to be matched and a matching degree value of the corresponding block groups, respectively.

8. A computer device, comprising: The computer device comprises: a memory storing executable program codes; a processor connected with the memory; the processor invokes the executable program codes stored in the memory to execute the image matching method of non-equivalent pixels according to any one of claims 1-5.

9. A storage medium, characterized by Executable program codes executable by a processor are stored, and the program codes are used to execute the image matching method of non-equivalent pixels according to any one of claims 1-5.

Images