Image comparison method and device based on lightweight neural network, equipment and medium

Abstract

The invention belongs to the technical field of data processing, and discloses an image comparison method, device and equipment based on a lightweight neural network, and a medium, and the method comprises the steps: obtaining a pre-trained neural network model, and carrying out the quantification processing; constructing a mobile terminal reasoning framework based on the quantized neural network model, and loading the quantized neural network model to a mobile terminal device through an interpreter; obtaining a first image and a second image to be compared and performing data preprocessing; respectively inputting the first image and the second image to be compared after data preprocessing into the neural network model after quantization processing for feature extraction; and calculating the cosine similarity between the first feature vector and the second feature vector, and judging the similarity between the first image to be compared and the second image to be compared according to the calculated cosine similarity. The method is suitable for business scenes of financial science and technology, medical health, old-age care and the like, and consumption of computing resources is effectively reduced when the mobile terminal carries out image comparison.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to an image comparison method, device, equipment and medium based on a lightweight neural network. Background Art

[0002] With the widespread adoption of mobile devices across various industries, image comparison technology has gradually become a core capability in intelligent scenarios. In finance and insurance, for example, during vehicle claims processing, personnel can quickly compare vehicle images using their mobile phones to improve claims processing efficiency. In healthcare and other fields, such as mobile medical testing equipment, image comparison technology is used to analyze medical images, facilitating early diagnosis and development of treatment plans.

[0003] Currently, commonly used image comparison methods include histogram comparison, structural similarity comparison, feature point matching algorithms, and convolutional neural network algorithms. However, each of these methods has limitations when used for image comparison. Histogram comparison has low accuracy in processing geometric changes in images, can only reflect color distribution, and is insensitive to structural differences. Structural similarity performs poorly when faced with a wide range of geometric transformations and has high computational complexity, making it unsuitable for real-time mobile applications. Feature point matching algorithms have high computational overhead and rely on powerful local or server-side computing resources, making their implementation and optimization on mobile devices more complex. While convolutional neural network algorithms can extract rich image features, their models are typically large, and loading and running pre-trained models requires high memory and computing resources, increasing the difficulty of mobile deployment.

[0004] In view of this, how to effectively reduce computing resource consumption when performing image comparison on mobile terminals is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The present invention provides an image comparison method, device, equipment and medium based on a lightweight neural network to solve the technical problem that existing image comparison methods consume a lot of resources when performing image comparison on a mobile terminal.

[0006] In a first aspect, the present invention provides an image comparison method based on a lightweight neural network, comprising:

[0007] Obtaining a pre-trained neural network model and performing quantization processing on the neural network model;

[0008] Building a mobile terminal inference framework based on the quantized neural network model, and loading the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal inference framework;

[0009] Acquire a first image and a second image to be compared from a mobile terminal device, and perform data preprocessing on the first image and the second image to be compared;

[0010] Inputting the first image and the second image to be compared after data preprocessing into the neural network model after quantization processing to perform feature extraction, thereby obtaining a first feature vector and a second feature vector of the first image and the second image to be compared;

[0011] The cosine similarity between the first eigenvector and the second eigenvector is calculated, and the similarity between the first image to be compared and the second image is determined based on the calculated cosine similarity.

[0012] In a second aspect, the present invention provides an image comparison device based on a lightweight neural network, wherein the device is used to implement the image comparison method based on a lightweight neural network as described in any one of the first aspects above, including:

[0013] A quantization module, used to obtain a pre-trained neural network model and perform quantization processing on the neural network model;

[0014] A loading module is used to build a mobile terminal reasoning framework based on the quantized neural network model, and load the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal reasoning framework;

[0015] a processing module, which acquires a first image and a second image to be compared from a mobile terminal device, and performs data preprocessing on the first image and the second image to be compared;

[0016] An extraction module, which inputs the first image and the second image to be compared after data preprocessing into the neural network model after quantization processing to perform feature extraction, thereby obtaining a first feature vector and a second feature vector of the first image and the second image to be compared;

[0017] The calculation module calculates the cosine similarity between the first eigenvector and the second eigenvector, and determines the similarity between the first image to be compared and the second image according to the calculated cosine similarity.

[0018] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned image comparison method based on a lightweight neural network are implemented.

[0019] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned image comparison method based on lightweight neural network.

[0020] The above-mentioned image comparison method, device, equipment and medium based on lightweight neural network, in the implemented solution, can obtain a pre-trained neural network model through the client and quantize the neural network model; build a mobile terminal inference framework based on the quantized neural network model, and load the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal inference framework; obtain a first image and a second image to be compared from the mobile terminal device, and preprocess the first and second images to be compared; input the first and second images to be compared after data preprocessing into the quantized neural network model for feature extraction, thereby obtaining a first feature vector and a second feature vector of the first and second images to be compared; calculate the cosine similarity between the first and second feature vectors, and judge the similarity between the first and second images to be compared based on the calculated cosine similarity. In the present invention, by quantizing the pre-trained neural network model, the storage space of the model on the mobile terminal is reduced, and the computational complexity of the model during inference is reduced, effectively reducing the computational resource consumption when performing image comparison on the mobile terminal. In addition, the quantized neural network model is loaded into the mobile terminal device through the interpreter, ensuring that the model can run efficiently on the mobile device. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 2 is a schematic diagram of an application environment of an image comparison method based on a lightweight neural network according to an embodiment of the present invention;

[0023] Figure 2 1 is a flow chart of an image comparison method based on a lightweight neural network according to an embodiment of the present invention;

[0024] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S10;

[0025] Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S30;

[0026] Figure 5 yes Figure 2 A schematic flow chart of a specific implementation of step S40;

[0027] Figure 6 1 is a schematic structural diagram of an image comparison device based on a lightweight neural network in one embodiment of the present invention;

[0028] Figure 7 is a structural diagram of a computer device in one embodiment of the present invention;

[0029] Figure 8 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] The image comparison method based on lightweight neural network provided by the embodiment of the present invention can be applied in the following fields: Figure 1 In the application environment, Figure 1 The present invention is a schematic diagram of an application environment of an image comparison method based on a lightweight neural network in an embodiment of the present invention; wherein the client communicates with the server through a network. The server can obtain a pre-trained neural network model through the client and quantize the neural network model; construct a mobile terminal inference framework based on the quantized neural network model, and load the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal inference framework; obtain a first image and a second image to be compared from the mobile terminal device, and perform data preprocessing on the first image and the second image to be compared; input the first image and the second image to be compared after data preprocessing into the quantized neural network model respectively for feature extraction to obtain a first feature vector and a second feature vector of the first image and the second image to be compared; calculate the cosine similarity between the first feature vector and the second feature vector, and judge the similarity of the first image and the second image to be compared based on the calculated cosine similarity. In the present invention, by quantizing the pre-trained neural network model, the storage space of the model in the mobile terminal is reduced, and the computational complexity of the model during inference is reduced, effectively reducing the computational resource consumption when performing image comparison on the mobile terminal. In addition, the quantized neural network model is loaded into the mobile terminal device through the interpreter, ensuring that the model can run efficiently on the mobile device. The present invention is described in detail below through specific embodiments.

[0032] See also Figure 2 As shown, Figure 2 A flowchart of a lightweight neural network-based image comparison method according to an embodiment of the present invention is provided. The lightweight neural network-based image comparison method specifically includes the following steps:

[0033] S10: Obtain a pre-trained neural network model and perform quantization processing on the neural network model.

[0034] When performing image comparison, the present invention first obtains a pre-trained neural network model. The pre-trained neural network model can be obtained by calling a model library, downloading a pre-trained model, or training on a specific task data set. Then, the pre-trained neural network model is quantized and processed into a low-bit neural network model using the tools provided by the model framework. For example, in a vehicle claims scenario, a convolutional neural network model pre-trained on large-scale road accident images is used to identify the damaged parts and degree of damage of the vehicle to be assessed. In order to quickly process the images of vehicles to be assessed on site, 8-bit quantization technology is used to reduce the model size and increase the inference speed. In medical scenarios, hospitals use deep models pre-trained on a large number of chest X-rays to detect lung diseases. In order to achieve rapid diagnosis, the model is quantized so that it can be deployed on a mobile device or portable image analyzer held by the doctor. Specifically, as Figure 3 As stated, Figure 3 This is a flow chart of a specific implementation of step S10, which specifically includes the following steps S11-S12:

[0035] S11: Obtain a pre-trained convolutional neural network model, and dynamically configure the quantization range of the pre-trained convolutional neural network model. Specifically, in an embodiment of the present invention, a pre-trained convolutional neural network model can be obtained first, for example, a pre-trained ResNet model and a MobileNet model can be obtained. The obtained pre-trained convolutional neural network model can be trained on a large-scale data set and has good feature extraction capabilities. Then, a "dynamic quantization" strategy is adopted to dynamically configure the quantization range of the pre-trained convolutional neural network model, which can ensure that the model can obtain good accuracy and compression effects under different input conditions. For example, in vehicle claims, by dynamically configuring the quantization range of the pre-trained convolutional neural network model and collecting vehicle images at different lighting and angles on the scene, it can be ensured that the model can still accurately identify different types of vehicle damage after compression. In medical scenarios, hospitals use pre-trained medical image analysis models and dynamically adjust the model activation range, so that the pre-trained convolutional neural network model can maintain good performance in images of different patients.

[0036] S12: The pre-trained convolutional neural network model is compressed according to the dynamically configured quantization range to obtain a lightweight convolutional neural network model. Specifically, in an embodiment of the present invention, the pre-trained convolutional neural network model can be compressed according to the dynamically configured quantization range in step S12 to obtain a lightweight convolutional neural network model. For example, in vehicle claims, when a mobile device is used on-site to quickly identify vehicle damage, the lightweight convolutional neural network model can achieve immediate reasoning and reduce network dependence. In medical diagnosis, medical mobile devices or portable diagnostic instruments need to run efficient models. The compressed model can run stably under limited hardware resources and support real-time diagnosis and remote consultation.

[0037] S20: Constructing a mobile terminal reasoning framework based on the quantized neural network model, and loading the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal reasoning framework. After the present invention quantizes the neural network model in step S10, it is necessary to construct a mobile terminal reasoning framework based on the quantized neural network model, select a reasoning engine suitable for the mobile terminal, integrate the corresponding reasoning library on the mobile device, convert the quantized model into a .tflite file, and embed the reasoning engine in the mobile terminal application. When the application is started or triggered by demand, the quantized model is loaded into the memory of the mobile terminal device through the interpreter of the reasoning framework.

[0038] S30: Obtain the first image and the second image to be compared from the mobile terminal device, and perform data preprocessing on the first image and the second image to be compared. Obtain the first image and the second image to be compared uploaded by the user or taken on-site through the camera or gallery interface of the mobile device. Specifically, Figure 4 As stated, Figure 4 This is a flow chart of a specific implementation of step S20, which specifically includes the following steps S31-S33:

[0039] S31: Convert the first image and the second image to be compared into BGR three-channel matrices respectively, and obtain the first image and the second image converted into BGR three-channel matrices. Specifically, in an embodiment of the present invention, when performing data preprocessing on the first image and the second image to be compared, first, the first image and the second image to be compared need to be converted into a BGR three-channel matrix. If the original image is in RGB format, the channel order is simply exchanged; if it is a grayscale image, it is necessary to copy the single channel to three channels to form a color matrix to ensure that the channel format of the two images is consistent, in preparation for subsequent unified preprocessing and feature extraction. For example, in a vehicle claims scenario, insurance staff use mobile phones to take photos of accident vehicles. The uploaded images may be in different formats and need to be converted into a BGR three-channel matrix to ensure that the model input is consistent, which is convenient for batch processing and model reasoning; in medical scenarios, medical images are usually grayscale images. By copying the grayscale single channel to three channels, the input format is ensured to be unified.

[0040] S32: Perform image preprocessing on the first image and the second image converted into the BGR three-channel matrix to obtain the first image and the second image after image preprocessing. Specifically, in an embodiment of the present invention, after obtaining the first image and the second image converted into the BGR three-channel matrix, it is also necessary to perform image preprocessing on the first image and the second image converted into the BGR three-channel matrix. For example, in a vehicle claims scenario, the vehicle damage photos may be different due to shooting angles and lighting. Image preprocessing is required for the first image and the second image to ensure that the two images are consistent in size and pixel range. In a medical scenario, the sizes of images taken at different times may be different. Preprocessing ensures that the two images have the same scale and brightness range to facilitate subsequent analysis. Among them, step S32 specifically includes:

[0041] S321: Convert the color space of the first image and the second image converted into a BGR three-channel matrix into an RGB three-channel matrix to obtain the first image and the second image converted into an RGB three-channel matrix. Specifically, in an embodiment of the present invention, the deep learning model usually requires the input image to be in RGB format. Therefore, it is necessary to convert the color space of the first image and the second image into an RGB three-channel matrix. For example, in a vehicle claims scenario, vehicle photos at the accident scene may be captured by different cameras and the saved formats are not uniform. It is necessary to convert the captured vehicle photos into a unified RGB three-channel format.

[0042] S322: The first image and the second image converted into the RGB three-channel matrix are resized to obtain the resized first image and the second image. Specifically, in an embodiment of the present invention, the scales of the first image and the second image can be adjusted to a uniform size. For example, the first image and the second image can be uniformly adjusted to 224*224. For example, in a vehicle claims scenario, vehicle images from different sources may have different resolutions. Resizing and unifying the sizes facilitates automatic vehicle damage detection and matching.

[0043] S323: Normalize the resized first image and the second image to obtain the normalized first image and the second image. Specifically, in an embodiment of the present invention, after obtaining the resized first image and the second image, it is necessary to further normalize the resized first image and the second image, and adjust the pixel range of the first image and the second image from integers [0-255] to floating-point numbers [0.0-1.0] to eliminate the influence of different image brightness and contrast. For example, in a vehicle claims scenario, normalizing the vehicle image can reduce the influence of light changes on the identification of damaged areas and improve the consistency of damage identification.

[0044] S324: Normalize the normalized first and second images to obtain the normalized first and second images. Specifically, in an embodiment of the present invention, after obtaining the normalized first and second images, each color channel of the first and second images can be normalized and scaled using a normalization formula. For example, in a vehicle claims scenario, normalization can make different vehicle images more comparable in the feature space, helping the model distinguish the severity or type of damage.

[0045] In one embodiment, the formula for normalizing the normalized first image and the second image is:

[0046]

[0047] Among them, [b R ,b G ,b B ] represents the normalized pixel value of each channel of the first and second images, [a R ,a G ,a B ] represents the actual pixel value of each channel of the first image and the second image, [μ R ,μ G ,μ B ] represents the mean pixel value of each channel of the first image and the second image, [σ R ,σ G ,σB ] represents the standard deviation pixel value of each channel of the first image and the second image.

[0048] S33: Performing shape transformation on the first image and the second image after image preprocessing to obtain the shape-transformed first image and the second image. Specifically, in an embodiment of the present invention, after obtaining the first image and the second image after image preprocessing, it is also necessary to change the array shape of the image, and adjust the array shape of the image to the input structure required by the model, so that the first image and the second image representation meet the input requirements of subsequent feature extraction or similarity calculation. For example, in a medical scenario, the two images are converted into vectors of the same dimension to facilitate subsequent input into a neural network model for feature extraction.

[0049] S40: Input the first image to be compared and the second image to be compared after data preprocessing into the neural network model after quantization processing to extract features, and obtain the first feature vector and the second feature vector of the first image to be compared and the second image. Figure 5 As stated, Figure 5 yes Figure 2 A flow chart of a specific implementation of step S40 in FIG. 4 includes the following steps S41-S43:

[0050] S41: The first image and the second image to be compared after data preprocessing are input into the initial convolution layer of the lightweight convolutional neural network model for shallow feature extraction to obtain preliminary feature maps of the first image and the second image. Specifically, in an embodiment of the present invention, after preprocessing, an image suitable for model input is obtained, and the first image and the second image to be compared can be first input into the first convolution layer of the model to extract low-level features, such as edge, texture, color and other feature information, and output as a preliminary feature map. For example, in a vehicle claims scenario, comparing photos of the vehicle before and after damage at the accident scene, first extracting shallow features of the photos can help the model quickly capture the edge and texture changes of the damaged area, providing a basis for subsequent deep analysis. In a medical scenario, comparing the differences between two scans or images, shallow feature extraction helps to highlight basic features such as structural contours and boundaries, providing clues for diagnosis.

[0051] S42: The preliminary feature maps of the first image and the second image are input into the depth separable layer of the lightweight convolutional neural network model for deep feature abstraction to obtain the depth feature maps of the first image and the second image. Specifically, in an embodiment of the present invention, the depth separable layer performs feature abstraction, which can separate the spatial dimension and the channel dimension, effectively reducing the number of parameters and the amount of calculation. For example, in a vehicle claims scenario, through deep features, the model can better understand the specific type of vehicle damage and assist in judging the severity of the damage. In a medical scenario, extracting lesion features in medical images can improve the accuracy of diagnosis.

[0052] S43: Input the depth feature maps of the first image and the second image into the attention mechanism layer of the lightweight convolutional neural network model for weighted processing to obtain the first feature vector and the second feature vector of the first image and the second image to be compared. Specifically, in an embodiment of the present invention, after obtaining the depth feature maps of the first image and the second image, the depth feature maps of the first image and the second image can be input into the attention mechanism layer of the lightweight convolutional neural network model for weighted processing to highlight key areas or important features. For example, in vehicle claims, the model can automatically focus on the damaged parts of the vehicle, ignore irrelevant background, and improve the accuracy of damage matching and similarity determination.

[0053] S50: Calculate the cosine similarity between the first feature vector and the second feature vector, and determine the similarity between the first image and the second image to be compared based on the calculated cosine similarity. In an embodiment of the present invention, the similarity determination mechanism based on cosine similarity can achieve automated and accurate image comparison, specifically including:

[0054] S51: A cosine similarity threshold is preset to indicate that the first image to be compared and the second image are similar, and the calculated cosine similarity between the first eigenvector and the second eigenvector is compared with the preset cosine similarity threshold. Specifically, in an embodiment of the present invention, a cosine similarity threshold can be preset to indicate that the first image to be compared and the second image are similar. For example, the cosine similarity threshold can be set to 0.8. By calculating whether the cosine similarity between the first eigenvector and the second eigenvector is greater than the preset cosine similarity threshold, an automated and accurate image comparison can be achieved.

[0055] S52: If the calculated cosine similarity is greater than or equal to the preset cosine similarity threshold, the first image to be compared and the second image are determined to be similar; if the calculated cosine similarity is less than the preset cosine similarity threshold, the first image to be compared and the second image are determined to be dissimilar. Specifically, in an embodiment of the present invention, if the cosine similarity threshold is set to 0.8 and the calculated cosine similarity is 0.9, the first image to be compared and the second image are determined to be similar; if the calculated cosine similarity is 0.5, the first image to be compared and the second image are determined to be dissimilar. For example, in a vehicle claims scenario, if the similarity between the two images of vehicle damage is high, it can be automatically confirmed that the damage belongs to the same vehicle or the same type, which can effectively improve the processing efficiency of vehicle claims. In a medical scenario, if the similarity between the captured images is high, it means that the structure or lesion features are consistent, which can help doctors confirm the type of disease or the effect of treatment.

[0056] As can be seen, in the above solution, by quantizing the pre-trained neural network model during image comparison, the model's storage space on the mobile terminal is reduced, while the computational complexity of the model during inference is also lowered, effectively reducing the computing resource consumption during image comparison on the mobile terminal. Furthermore, loading the quantized neural network model onto the mobile terminal device through an interpreter ensures that the model can run efficiently on the mobile device.

[0057] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0058] In one embodiment, a lightweight neural network-based image comparison device is provided, which corresponds one-to-one to the lightweight neural network-based image comparison method in the above embodiment. Figure 6 As shown, Figure 6 FIG6 is a schematic diagram of a structure of an image comparison device based on a lightweight neural network according to an embodiment of the present invention. The image comparison device based on a lightweight neural network includes a quantization module 61, a loading module 62, a processing module 63, an extraction module 64, and a calculation module 65. The functional modules are described in detail as follows:

[0059] A quantization module 61 is used to obtain a pre-trained neural network model and perform quantization processing on the neural network model;

[0060] A loading module 62 is configured to construct a mobile terminal reasoning framework based on the quantized neural network model and load the quantized neural network model into the mobile terminal device through an interpreter of the mobile terminal reasoning framework;

[0061] The processing module 63 acquires the first image and the second image to be compared from the mobile terminal device, and performs data preprocessing on the first image and the second image to be compared;

[0062] An extraction module 64 inputs the first image and the second image to be compared after data preprocessing into the neural network model after quantization processing to perform feature extraction, thereby obtaining a first feature vector and a second feature vector of the first image and the second image to be compared;

[0063] The calculation module 65 calculates the cosine similarity between the first eigenvector and the second eigenvector, and determines the similarity between the first image to be compared and the second image according to the calculated cosine similarity.

[0064] In one embodiment, the quantization module 61 is specifically configured to:

[0065] Obtaining a pre-trained convolutional neural network model, and dynamically configuring a quantization range of the pre-trained convolutional neural network model;

[0066] The pre-trained convolutional neural network model is compressed according to the dynamically configured quantization range to obtain a lightweight convolutional neural network model.

[0067] In one embodiment, the processing module 63 is specifically configured to:

[0068] Converting the first image and the second image to be compared into BGR three-channel matrices respectively to obtain the first image and the second image converted into BGR three-channel matrices;

[0069] Performing image preprocessing on the first image and the second image converted into the BGR three-channel matrix to obtain the first image and the second image after image preprocessing;

[0070] The first image and the second image after image preprocessing are subjected to shape transformation processing to obtain the first image and the second image after shape transformation.

[0071] In one embodiment, the processing module 63 is specifically configured to:

[0072] Convert the color spaces of the first image and the second image converted into the BGR three-channel matrix into an RGB three-channel matrix to obtain the first image and the second image converted into the RGB three-channel matrix;

[0073] Performing a size transformation on the first image and the second image converted into the RGB three-channel matrix to obtain the size-transformed first image and the second image;

[0074] Normalizing the resized first image and the second image to obtain normalized first and second images;

[0075] The normalized first image and the second image are normalized to obtain the normalized first image and the second image.

[0076] In one embodiment, the processing module 63 is specifically configured to:

[0077] The formula for normalizing the normalized first image and the second image is:

[0078]

[0079] Among them, [b R ,b G ,b B ] represents the normalized pixel values ​​of each channel of the first and second images,

[0080] [aR ,a G ,a B ] represents the actual pixel value of each channel of the first image and the second image, [μ R ,μ G ,μ B ] represents the mean pixel value of each channel of the first image and the second image, [σ R ,σ G ,σ B ] represents the standard deviation pixel value of each channel of the first image and the second image.

[0081] In one embodiment, the extraction module 64 is specifically configured to:

[0082] Inputting the first image and the second image to be compared after data preprocessing into the initial convolution layer of the lightweight convolutional neural network model for shallow feature extraction to obtain preliminary feature maps of the first image and the second image;

[0083] Inputting the preliminary feature maps of the first image and the second image into the depth-separable layer of the lightweight convolutional neural network model for deep feature abstraction to obtain deep feature maps of the first image and the second image;

[0084] The depth feature maps of the first image and the second image are input into the attention mechanism layer of the lightweight convolutional neural network model for weighted processing to obtain the first feature vector and the second feature vector of the first image and the second image to be compared.

[0085] In one embodiment, the calculation module 65 is specifically configured to:

[0086] Presetting a cosine similarity threshold value at which the first image and the second image to be compared are similar, and comparing the calculated cosine similarity between the first eigenvector and the second eigenvector with the preset cosine similarity threshold value;

[0087] If the calculated cosine similarity is greater than or equal to a preset cosine similarity threshold, it is determined that the first image to be compared and the second image are similar;

[0088] If the calculated cosine similarity is less than a preset cosine similarity threshold, it is determined that the first image to be compared and the second image are not similar.

[0089] This invention provides an image comparison device based on a lightweight neural network. By quantizing a pre-trained neural network model during image comparison, this device reduces the model's storage space on the mobile terminal and the computational complexity of the model during inference, effectively reducing computing resource consumption during image comparison on the mobile terminal. Furthermore, an interpreter is used to load the quantized neural network model onto the mobile terminal device, ensuring that the model can run efficiently on the mobile device.

[0090] For the specific definition of the image comparison device based on a lightweight neural network, please refer to the definition of the image comparison method based on a lightweight neural network above, and will not be repeated here. The various modules in the above-mentioned image comparison device based on a lightweight neural network can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above-mentioned modules.

[0091] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, Figure 7 1 is a structural diagram of a computer device in one embodiment of the present invention. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of an image comparison method based on a lightweight neural network.

[0092] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 8 As shown, Figure 8 1 is another structural schematic diagram of a computer device in one embodiment of the present invention. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a lightweight neural network-based image comparison method.

[0093] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0094] Obtaining a pre-trained neural network model and performing quantization processing on the neural network model;

[0095] Building a mobile terminal inference framework based on the quantized neural network model, and loading the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal inference framework;

[0096] Acquire a first image and a second image to be compared from a mobile terminal device, and perform data preprocessing on the first image and the second image to be compared;

[0097] Inputting the first image and the second image to be compared after data preprocessing into the neural network model after quantization processing to perform feature extraction, thereby obtaining a first feature vector and a second feature vector of the first image and the second image to be compared;

[0098] The cosine similarity between the first eigenvector and the second eigenvector is calculated, and the similarity between the first image to be compared and the second image is determined based on the calculated cosine similarity.

[0099] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0100] Obtaining a pre-trained neural network model and performing quantization processing on the neural network model;

[0101] Building a mobile terminal inference framework based on the quantized neural network model, and loading the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal inference framework;

[0102] Acquire a first image and a second image to be compared from a mobile terminal device, and perform data preprocessing on the first image and the second image to be compared;

[0103] Inputting the first image and the second image to be compared after data preprocessing into the neural network model after quantization processing to perform feature extraction, thereby obtaining a first feature vector and a second feature vector of the first image and the second image to be compared;

[0104] The cosine similarity between the first eigenvector and the second eigenvector is calculated, and the similarity between the first image to be compared and the second image is determined based on the calculated cosine similarity.

[0105] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0106] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0107] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0108] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. An image comparison method based on a lightweight neural network, characterized in that: Applicable to mobile terminal devices, including: Obtaining a pre-trained neural network model and performing quantization processing on the neural network model; Building a mobile terminal inference framework based on the quantized neural network model, and loading the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal inference framework; Acquire a first image and a second image to be compared from a mobile terminal device, and perform data preprocessing on the first image and the second image to be compared; Inputting the first image and the second image to be compared after data preprocessing into the neural network model after quantization processing to perform feature extraction, thereby obtaining a first feature vector and a second feature vector of the first image and the second image to be compared; The cosine similarity between the first eigenvector and the second eigenvector is calculated, and the similarity between the first image to be compared and the second image is determined based on the calculated cosine similarity.

2. The image comparison method based on lightweight neural network according to claim 1, characterized in that: The obtaining of a pre-trained neural network model and performing quantization processing on the neural network model includes: Obtaining a pre-trained convolutional neural network model, and dynamically configuring a quantization range of the pre-trained convolutional neural network model; The pre-trained convolutional neural network model is compressed according to the dynamically configured quantization range to obtain a lightweight convolutional neural network model.

3. The image comparison method based on lightweight neural network according to claim 1, characterized in that: The step of acquiring a first image and a second image to be compared from a mobile terminal device and performing data preprocessing on the first image and the second image includes: Converting the first image and the second image to be compared into BGR three-channel matrices respectively to obtain the first image and the second image converted into BGR three-channel matrices; Performing image preprocessing on the first image and the second image converted into the BGR three-channel matrix to obtain the first image and the second image after image preprocessing; The first image and the second image after image preprocessing are subjected to shape transformation processing to obtain the first image and the second image after shape transformation.

4. The image comparison method based on lightweight neural network according to claim 3, characterized in that: The performing image preprocessing on the first image and the second image converted into the BGR three-channel matrix to obtain the first image and the second image after image preprocessing includes: Convert the color spaces of the first image and the second image converted into the BGR three-channel matrix into an RGB three-channel matrix to obtain the first image and the second image converted into the RGB three-channel matrix; Performing a size transformation on the first image and the second image converted into the RGB three-channel matrix to obtain the size-transformed first image and the second image; Normalizing the resized first image and the second image to obtain normalized first and second images; The normalized first image and the second image are normalized to obtain the normalized first image and the second image.

5. The image comparison method based on lightweight neural network according to claim 1, characterized in that: The step of performing normalization on the normalized first image and the second image to obtain the normalized first image and the second image comprises: The formula for normalizing the normalized first image and the second image is: Among them, [b R ,b G ,b B ] represents the normalized pixel value of each channel of the first and second images, [a R ,a G ,a B ] represents the actual pixel value of each channel of the first image and the second image, [μ R ,μ G ,μ B ] represents the mean pixel value of each channel of the first image and the second image, [σ R ,σ G ,σ B ] represents the standard deviation pixel value of each channel of the first image and the second image.

6. The image comparison method based on lightweight neural network according to claim 2, characterized in that: The first image and the second image to be compared after data preprocessing are input into the neural network model after quantization processing to respectively extract features to obtain first feature vectors and second feature vectors of the first image and the second image to be compared, including: The first image and the second image to be compared after data preprocessing are input into the initial convolution layer of the lightweight convolutional neural network model for shallow feature extraction to obtain preliminary feature maps of the first image and the second image; Inputting the preliminary feature maps of the first image and the second image into the depth-separable layer of the lightweight convolutional neural network model for deep feature abstraction to obtain deep feature maps of the first image and the second image; The depth feature maps of the first image and the second image are input into the attention mechanism layer of the lightweight convolutional neural network model for weighted processing to obtain the first feature vector and the second feature vector of the first image and the second image to be compared.

7. The image comparison method based on lightweight neural network according to claim 1, characterized in that: The calculating the cosine similarity between the first eigenvector and the second eigenvector, and judging the similarity between the first image to be compared and the second image according to the calculated cosine similarity, includes: Presetting a cosine similarity threshold value at which the first image and the second image to be compared are similar, and comparing the calculated cosine similarity between the first eigenvector and the second eigenvector with the preset cosine similarity threshold value; If the calculated cosine similarity is greater than or equal to a preset cosine similarity threshold, it is determined that the first image to be compared and the second image are similar; If the calculated cosine similarity is less than a preset cosine similarity threshold, it is determined that the first image to be compared and the second image are not similar.

8. An image comparison device based on a lightweight neural network, characterized in that: The device is used to implement the image comparison method based on a lightweight neural network according to any one of claims 1 to 7, comprising: A quantization module, used to obtain a pre-trained neural network model and perform quantization processing on the neural network model; A loading module is used to build a mobile terminal reasoning framework based on the quantized neural network model, and load the quantized neural network model into the mobile terminal device through the interpreter of the mobile terminal reasoning framework; a processing module, which acquires a first image and a second image to be compared from a mobile terminal device, and performs data preprocessing on the first image and the second image to be compared; An extraction module, which inputs the first image and the second image to be compared after data preprocessing into the neural network model after quantization processing to perform feature extraction, thereby obtaining a first feature vector and a second feature vector of the first image and the second image to be compared; The calculation module calculates the cosine similarity between the first eigenvector and the second eigenvector, and determines the similarity between the first image to be compared and the second image according to the calculated cosine similarity.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the image comparison method based on a lightweight neural network are implemented as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the image comparison method based on a lightweight neural network are implemented as claimed in any one of claims 1 to 7.

Images