Image difference detection method and device, program product and electronic equipment

Abstract

The invention discloses an image difference detection method and device, a program product and electronic equipment. The method comprises the following steps: pre-detecting a first image and a second image; under the condition that the pre-detection result is a preset result, obtaining target mapping; feature extraction is carried out on the first image and the second image based on target mapping, a first matrix and a second matrix are obtained, the first matrix is used for representing region features of the maximum rectangular region matched by the first key point in the first image, and the second matrix is used for representing region features of the maximum rectangular region matched by the second key point in the second image; the second matrix is used for representing region features of the maximum rectangular region matched by the second key point in the second image; and determining a target coordinate according to the first matrix, the second matrix and the target mapping. According to the method and the device, the technical problem of low difference detection accuracy caused by dislocation of backgrounds of the two images participating in detection due to jitter of the image acquisition equipment when difference detection is carried out on the two images shot by the same image acquisition equipment in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an image difference detection method, device, program product, and electronic device. Background Art

[0002] In the business scenario of monitoring changes in natural resources, in order to overcome the problems of long detection cycles and low accuracy in detecting resource changes in a small area when performing difference detection based on high-resolution remote sensing image comparison, existing technologies can use cameras to periodically capture and compare natural resource images, thereby detecting changes in natural resources in a small area within a shorter period. This technology can serve as a supplement to monitoring changes in natural resources based on remote sensing images.

[0003] In the prior art, cameras used for monitoring changes in natural resources are usually installed on construction equipment such as towers. Ideally, technical personnel in this field can obtain two images to be compared at different times with completely consistent parameters related to the camera posture. However, in real scenarios, due to camera control issues, when comparing images taken by the same camera at different times, the camera control will be affected by multiple factors, causing the shooting angle to shake, resulting in the two images to be compared not being taken by the same camera with the same posture, that is, there will be a certain misalignment in the backgrounds of the two images, and the field of view of the two images to be compared will be misaligned, which will lead to a technical problem of low accuracy in image difference detection.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The present application provides an image difference detection method, apparatus, program product, and electronic device to at least address the technical problem in the prior art of low difference detection accuracy caused by misalignment of the backgrounds of the two images participating in the detection due to jitter of the image acquisition device when performing difference detection on two images taken by the same image acquisition device.

[0006] According to one aspect of the present application, an image difference detection method is provided, comprising: performing pre-detection on a first image and a second image, wherein the pre-detection is at least used to directly perform difference detection based on image matrices corresponding to the two images; obtaining a target mapping when the pre-detection result is a preset result, wherein the target mapping is used to characterize the matching result between a first key point and a second key point, the first key point being a pixel point in the first image, and the second key point being a pixel point in the second image, and the preset result being used to characterize the presence of potential difference points between the first image and the second image; performing feature extraction on the first image and the second image based on the target mapping to obtain a first matrix and a second matrix, wherein the first matrix is used to characterize regional features of the largest rectangular area matched by the first key point in the first image, and the second matrix is used to characterize regional features of the largest rectangular area matched by the second key point in the second image; and determining target coordinates based on the first matrix, the second matrix, and the target mapping, wherein the target coordinates are used to characterize the actual positions of the difference points corresponding to the first image and the second image.

[0007] Optionally, before pre-detecting the first image and the second image, the image difference detection method further includes: acquiring two original images, wherein the two original images are images captured by the same image acquisition device at different times; pre-processing the two original images to obtain the first image and the second image, wherein the pre-processing is at least used to perform grayscale processing and size alignment processing on the two original images.

[0008] Optionally, pre-detection is performed on the first image and the second image, including: performing feature extraction on the first image to obtain a first image matrix, wherein the first image matrix is used to characterize local features of key points included in the first image; performing feature extraction on the second image to obtain a second image matrix, wherein the second image matrix is used to characterize local features of key points included in the second image; and pre-detection is performed on the first image and the second image based on the first image matrix and the second image matrix.

[0009] Optionally, determining the target coordinates based on the first matrix, the second matrix and the target mapping includes: obtaining a first difference matrix corresponding to the first matrix and the second matrix, wherein the values in the first difference matrix are used to represent the distance between the SIFT descriptors included in the first matrix and the SIFT descriptors included in the second matrix; updating the size of the first difference matrix to obtain a second difference matrix, wherein the size of the second difference matrix is equal to the original size of the first image, and the original size of the first image is the size of the first image before preprocessing; determining the target difference contour based on the second difference matrix, wherein the target difference contour is the contour corresponding to the difference points included in the second difference matrix; and determining the target coordinates based on the target difference contour and the target mapping.

[0010] Optionally, determining the target coordinates based on the target difference contour and the target mapping includes: determining a first rectangular box based on the circumscribed matrix coordinates corresponding to the target difference contour, wherein the first rectangular box is used to represent the potential position of the difference point in the first image; determining a second rectangular box based on the circumscribed matrix coordinates corresponding to the target difference contour and the target mapping, wherein the second rectangular box is used to represent the potential position of the difference point in the second image; and determining the target coordinates based on the first rectangular box and the second rectangular box.

[0011] Optionally, determining the target coordinates based on the first rectangular frame and the second rectangular frame includes: segmenting the first image based on the first rectangular frame to obtain a first sub-image; segmenting the second image based on the second rectangular frame to obtain a second sub-image; obtaining the structural similarity between the first sub-image and the second sub-image; and when the structural similarity is greater than a preset threshold, using the coordinates of the first rectangular frame and the coordinates of the second rectangular frame as the target coordinates.

[0012] Optionally, before performing feature extraction on the first image, the image difference detection method further includes: obtaining a camera height and a camera focal length, wherein the camera height is the height at which the image acquisition device is installed when acquiring the original image, and the camera focal length is the focal length set by the image acquisition device when acquiring the original image; determining a first parameter and a second parameter based on the camera height and the camera focal length, wherein the first parameter is used to determine the density of the matrix points generated for feature extraction of the image, and the second parameter is an area ratio filtering threshold corresponding to the difference rectangular box, and the difference rectangular box includes a first rectangular box and a second rectangular box.

[0013] According to another aspect of the present application, an image difference detection device is also provided, including: a pre-detection unit, used to perform pre-detection on a first image and a second image, wherein the pre-detection is at least used to directly perform difference detection based on the image matrix corresponding to the two images; a first acquisition unit, used to obtain a target mapping when the pre-detection result is a preset result, wherein the target mapping is used to characterize the matching result between the first key point and the second key point, the first key point is a pixel point in the first image, and the second key point is a pixel point in the second image, and the preset result is used to characterize the presence of potential difference points between the first image and the second image; a feature extraction unit, used to perform feature extraction on the first image and the second image based on the target mapping to obtain a first matrix and a second matrix, wherein the first matrix is used to characterize the regional features of the largest rectangular area matched by the first key point in the first image, and the second matrix is used to characterize the regional features of the largest rectangular area matched by the second key point in the second image; a first determination unit, used to determine the target coordinates based on the first matrix, the second matrix and the target mapping, wherein the target coordinates are used to characterize the actual positions of the difference points corresponding to the first image and the second image.

[0014] According to another aspect of the present application, a computer program product is further provided. The computer program product stores a computer program, wherein when the computer program is run, the computer program product is controlled to execute any one of the above-mentioned image difference detection methods.

[0015] According to another aspect of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned image difference detection methods.

[0016] In the present application, the first image and the second image are first pre-detected, wherein the pre-detection is at least used to directly perform difference detection based on the image matrix corresponding to the two images. When the pre-detection result is a preset result, the present application obtains a target mapping, wherein the target mapping is used to characterize the matching result between the first key point and the second key point, the first key point is a pixel point in the first image, and the second key point is a pixel point in the second image. The preset result is used to characterize the potential difference point between the first image and the second image. Afterwards, the present application performs feature extraction on the first image and the second image based on the target mapping to obtain a first matrix and a second matrix, wherein the first matrix is used to characterize the regional features of the largest rectangular area matched by the first key point in the first image, and the second matrix is used to characterize the regional features of the largest rectangular area matched by the second key point in the second image. Then, the present application determines the target coordinates based on the first matrix, the second matrix and the target mapping, wherein the target coordinates are used to characterize the actual positions of the difference points corresponding to the first image and the second image.

[0017] From the above content, it can be seen that when detecting image differences, the present application first performs a pre-detection on the two images, that is, the difference detection is performed directly based on the image matrices corresponding to the two images with potential background misalignment. When the pre-detection shows that there is a potential difference between the two images, the present application performs a fine detection on the above two images, that is, first obtains the target mapping, and the target mapping is used to characterize the matching results between the key points included in the two images. Subsequently, the present application performs feature extraction on the two images respectively according to the target mapping, so as to obtain the regional features corresponding to the largest rectangular area matched by the key points in the image. After that, the difference detection is further performed based on the regional features corresponding to the key points in the two images (that is, the first matrix and the second matrix) and the target mapping to obtain the actual position of the difference point in the image.

[0018] It can be seen that the present application achieves the purpose of aligning the contrasting fields of view of two images with background misalignment by introducing target mapping in the fine detection step, thereby overcoming the influence of background misalignment on image difference detection, thereby achieving the technical effect of improving the accuracy of image difference detection, and further solving the technical problem in the prior art of low difference detection accuracy caused by the misalignment of the backgrounds of the two images participating in the detection due to the jitter of the image acquisition device when performing difference detection on two images taken by the same image acquisition device. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 is a flowchart of an optional image difference detection method according to an embodiment of the present application;

[0021] Figure 2 is a flowchart of an optional image difference detection algorithm according to an embodiment of the present application;

[0022] Figure 3 is a flowchart of an optional image difference pre-detection algorithm according to an embodiment of the present application;

[0023] Figure 4 is a flowchart of an optional image difference fine detection algorithm according to an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of an optional image difference detection device according to an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] It should also be noted that the relevant information (including but not limited to information for display and analysis) and data (including but not limited to image data) involved in this application are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set up between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0029] In addition, the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant information and data involved in this application comply with the relevant laws, regulations and standards of the relevant regions, and necessary confidentiality measures have been taken, and do not violate public order and good morals. In addition, this application provides corresponding operation entrances for users to choose to agree to authorization or refuse authorization. If the user chooses to refuse authorization, he / she will enter the corresponding expert decision-making process.

[0030] The relevant terms involved in this application are explained as follows:

[0031] Image comparison: Image comparison is one of the common tasks in computer vision and image processing. Image comparison can be used in application scenarios such as identifying duplicate images, image search, image similarity comparison, and identifying image differences. Image comparison can select appropriate solutions based on different detection requirements and image types. For example, for simple image comparison requirements, methods based on pixel comparison or histogram comparison can be selected, while for complex image comparison requirements, methods based on feature extraction and matching or deep learning models can be selected. Image comparison is currently widely used in scenarios such as identifying duplicate images, image search, and image similarity comparison. The above application scenarios only require whether two images are similar and how similar they are. Identifying image differences requires finding the locations of the different parts of the two images. For example, the computer "spot the difference" game can be achieved through image segmentation, grayscale conversion, binarization, and contour search technologies.

[0032] SIFT (Scale-Invariant Feature Transform Descriptor): The SIFT descriptor is constructed by establishing a description region around the key point and then calculating the directional histogram of the image gradient in the region. Finally, a fixed-length vector is generated. Each dimension of the vector represents a quantitative feature of the image gradient around the key point, which is used to describe the local structure of the key point.

[0033] According to an embodiment of the present application, an embodiment of an image difference detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] This application provides an image difference detection system (hereinafter referred to as the detection system) for executing the image difference detection method in this application. Figure 1 is a flow chart of an optional image difference detection method according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0035] Step S101: Pre-detection is performed on the first image and the second image.

[0036] In step S101 , the pre-detection is at least used to directly perform difference detection based on image matrices corresponding to two images.

[0037] Optionally, the first image and the second image are obtained by preprocessing original images captured at different times by the same image capture device installed on a high-rise building such as an iron tower, wherein the size of the first image is the same as the size of the second image.

[0038] Optionally, during the pre-detection process, by calculating the feature distance between the SIFT descriptors of the corresponding matrix points of the two images, the detection system can preliminarily determine whether the two images contain potential difference areas. The pre-detection is usually performed based on the global features of the image. The pre-detection directly obtains the difference rectangular box through matrix point difference analysis and similarity calculation, thereby quickly identifying blocks with larger difference areas or areas with smaller similarities, and then quickly locating the locations where potential differences exist.

[0039] Step S102: When the pre-detection result is a preset result, a target mapping is obtained.

[0040] In step S102, the target mapping is used to characterize the matching result between the first key point and the second key point, where the first key point is a pixel point in the first image and the second key point is a pixel point in the second image, and the preset result is used to characterize the potential difference between the first image and the second image.

[0041] Optionally, the detection system first matches all first key points included in the first image and all second key points included in the second image, and then extracts the homography transformation of the two images respectively based on the matching results between the first key points and the second key points, and establishes the homography transformation mapping relationship between the first image and the second image to obtain the target mapping.

[0042] Optionally, when the pre-detection result is not the preset result, the detection system determines that there is no potential difference between the first image and the second image, that is, there is no misalignment between the backgrounds of the first image and the second image. At this time, the detection system directly transmits the pre-detection result to the user, prompting that there is no difference between the first image and the second image.

[0043] From the above content, it can be seen that the detection system starts the second stage of fine detection (i.e., step S102 to step S104) on the first image and the second image only when there is a difference between the pre-detection results, thereby achieving the purpose of avoiding fine detection of two images that are not misaligned, reducing the amount of data required for fine detection in this application, and thereby improving the speed of detecting image differences. In other words, the purpose of pre-detection is to quickly filter out a large number of image pairs with no substantial differences, thereby reducing the amount of calculation of subsequent fine detection, and thereby improving the overall detection efficiency.

[0044] In addition, the above-mentioned target mapping represents the matching result between the first key point included in the first image and the second key point included in the second image. The detection system achieves the purpose of aligning the contrast field of view of the first image and the second image by establishing a mapping between the first key point and the second key point. This step can determine the correspondence between the two images under the factors of translation, rotation and size change, thereby eliminating the impact of background misalignment between the first image and the second image on image difference detection.

[0045] Step S103 : performing feature extraction on the first image and the second image based on the target mapping to obtain a first matrix and a second matrix.

[0046] In step S103 , the first matrix is used to characterize the regional features of the largest rectangular area matched by the first key point in the first image, and the second matrix is used to characterize the regional features of the largest rectangular area matched by the second key point in the second image.

[0047] Optionally, the detection system first generates matrix points based on the extracted target mapping. The range of the generated matrix is not for the first and second entire images, but for the largest rectangular area matched by the key points in the image, and the SIFT descriptors of the matrix points corresponding to the two images are calculated respectively, thereby obtaining the first matrix and the second matrix.

[0048] Optionally, the detection system extracts features from the first image and the second image based on target mapping to obtain a first matrix representing regional features of the largest rectangular area matched by the first key point in the first image, and a second matrix representing regional features of the largest rectangular area matched by the second key point in the second image. Thereafter, the detection system performs image difference detection based on the first matrix and the second matrix. Compared with the pre-detection stage, the detection system focuses on the regional features corresponding to the largest rectangular area matched by the key point in the image in the current detection stage, rather than the full image features including redundant information such as the image background, thereby improving the accuracy of image difference detection.

[0049] Step S104 , determining target coordinates according to the first matrix, the second matrix, and the target mapping.

[0050] In step S104 , the target coordinates are used to represent the actual positions of the difference points corresponding to the first image and the second image.

[0051] Optionally, the detection system determines which local areas have significant structural differences based on the difference features of the first matrix and the second matrix in combination with target mapping. By calculating the similarity between the feature descriptors of the difference points and the non-difference points, the detection system can identify the difference points and map the difference points from the feature space back to the image space through target mapping, thereby determining the position coordinates of the difference points in the original image and obtaining the target coordinates.

[0052] Optionally, when the first image and the second image include more than one difference, the detection system outputs the target coordinates corresponding to each of the multiple differences.

[0053] From the above content, it can be seen that when detecting image differences, the present application first performs a pre-detection on the two images, that is, the difference detection is performed directly based on the image matrices corresponding to the two images with potential background misalignment. When the pre-detection shows that there is a potential difference between the two images, the present application performs a fine detection on the above two images, that is, first obtains the target mapping, and the target mapping is used to characterize the matching results between the key points included in the two images. Subsequently, the present application performs feature extraction on the two images respectively according to the target mapping, so as to obtain the regional features corresponding to the largest rectangular area matched by the key points in the image. After that, the difference detection is further performed based on the regional features corresponding to the key points in the two images (that is, the first matrix and the second matrix) and the target mapping to obtain the actual position of the difference point in the image.

[0054] It can be seen that the present application achieves the purpose of aligning the contrasting fields of view of two images with background misalignment by introducing target mapping in the fine detection step, thereby overcoming the influence of background misalignment on image difference detection, thereby achieving the technical effect of improving the accuracy of image difference detection, and further solving the technical problem in the prior art of low difference detection accuracy caused by the misalignment of the backgrounds of the two images participating in the detection due to the jitter of the image acquisition device when performing difference detection on two images taken by the same image acquisition device.

[0055] In an optional embodiment, the detection system first acquires two original images, wherein the two original images are images captured by the same image acquisition device at different times. Thereafter, the detection system preprocesses the two original images to obtain a first image and a second image, wherein the preprocessing is at least used to perform grayscale processing and size alignment processing on the two original images.

[0056] In the above embodiment, the grayscale processing aims to convert a color image into a grayscale image, thereby reducing the color variables in image processing, allowing the detection system to pay more attention to the structural features and brightness information of the image, and thus achieving the technical effect of eliminating the interference caused by color differences during the image comparison process and improving the accuracy of difference detection.

[0057] In the above embodiment, the size alignment process is to make the first image and the second image have the same size, so that the detection system can directly compare the first image and the second image at the pixel level, thereby avoiding comparison errors caused by different image sizes. For example, when the sizes of the two original images are inconsistent, the present application adjusts the two images to the same size through image scaling techniques (such as bilinear interpolation and nearest neighbor interpolation), thereby ensuring the consistency and effectiveness of subsequent processing.

[0058] In an optional embodiment, the detection system first performs feature extraction on the first image to obtain a first image matrix, wherein the first image matrix is used to characterize the local features of the key points included in the first image. Then, the detection system performs feature extraction on the second image to obtain a second image matrix, wherein the second image matrix is used to characterize the local features of the key points included in the second image. Then, the detection system performs preliminary detection on the first image and the second image based on the first image matrix and the second image matrix.

[0059] In the above embodiment, the step of extracting features from the two images during the pre-detection process includes:

[0060] (1) Key point detection: The DOG (Difference of Gaussians) method is used to detect key points in the first image and the second image.

[0061] (2) Key point positioning: Accurately locate the detected key points, remove points with too strong edge response in the image, and find more accurate key point positions through Taylor expansion, thereby improving the accuracy of key point positioning and reducing the probability of mismatching in subsequent calculations.

[0062] (3) Direction assignment: A dominant direction is assigned to each key point based on the gradient direction around the key point, thereby ensuring that even if the image is rotated, the descriptor of the key point can still match the descriptor before rotation, thereby increasing the robustness of the matching result.

[0063] (4) Scale-space key point descriptor generation: The SIFT algorithm is used to calculate the SIFT descriptor corresponding to the key point. The SIFT descriptor is a quantitative representation of the features of the area around the key point. The SIFT descriptor is a vector of fixed length that reflects the local appearance features of the area where the key point is located. The detection system uses the scale and direction information of the key point. By sampling and quantizing the gradient direction of the image around the key point, a SIFT descriptor that is invariant to scale and rotation is constructed.

[0064] (5) Feature point descriptor optimization: The detection system normalizes the generated descriptors to enhance the adaptability of the processed feature point descriptors under different lighting conditions, thereby improving the versatility and matching performance of the feature point descriptors.

[0065] (6) Feature vector storage: The detection system saves the descriptor information of each key point as a feature vector. These vectors constitute the feature set of the image, namely the first image matrix and the second image matrix in the pre-detection step.

[0066] Optionally, after obtaining the first image matrix and the second image matrix, the detection system compares the differences in SIFT descriptors of corresponding key points in the two matrices to screen out potential difference points. The detection system further calculates the structural similarity of these difference points to preliminarily determine the potential position of the difference area.

[0067] In an optional embodiment, the detection system first obtains a first difference matrix corresponding to the first matrix and the second matrix, wherein the values in the first difference matrix are used to characterize the distance between the SIFT descriptors included in the first matrix and the SIFT descriptors included in the second matrix. Afterwards, the detection system updates the size of the first difference matrix to obtain a second difference matrix, wherein the size of the second difference matrix is equal to the original size of the first image, and the original size of the first image is the size of the first image before preprocessing. Then, the detection system determines the target difference contour based on the second difference matrix, wherein the target difference contour is the contour corresponding to the difference points included in the second difference matrix. Finally, the detection system determines the target coordinates based on the target difference contour and the target mapping.

[0068] Optionally, after the detection system obtains the first matrix and the second matrix, the detection system will obtain the feature distance between the matrix point SIFT descriptors included in the first matrix and the corresponding matrix point SIFT descriptors in the second matrix, and take the matrix points whose feature distances are greater than the preset distance as difference points, thereby obtaining a first difference matrix, wherein the feature distance values between the matrix point descriptors reflect the degree of similarity between the matrix points, and the larger the distance, the greater the difference between the feature points.

[0069] Optionally, since the size of the first difference matrix does not match the size of the original image, the detection system needs to adjust the size of the first difference matrix to be equal to the original size of the first image, that is, perform reverse mapping to generate a second difference matrix, so that the second difference matrix can accurately reflect the change position of the feature points at the original image size, which facilitates subsequent visual understanding and difference point extraction.

[0070] Optionally, the detection system screens out a set of points that are continuous and have difference values in the second difference matrix based on connected component analysis technology or contour detection technology, thereby forming a difference contour. At the same time, the detection system can filter out contours that are tiny and have no substantial differences by setting the second parameter, thereby improving the accuracy of the difference detection results.

[0071] In the above embodiment, the detection system updates the size of the first difference matrix obtained by calculating the SIFT descriptor, thereby ensuring that the coordinates of the difference points can be accurately reflected in the original image, thereby enhancing the detection system's positioning ability for change detection. Furthermore, the detection system can identify the real changes of entities in the first image and the second image by determining the target difference contour and mapping the target coordinates, rather than mistakenly identifying false changes caused by image misalignment, thereby reducing the false detection probability of the detection system.

[0072] In an optional embodiment, the detection system first determines a first rectangular box based on the circumscribed matrix coordinates corresponding to the target difference contour, wherein the first rectangular box is used to represent the potential position of the difference point in the first image. Then, the detection system determines a second rectangular box based on the circumscribed matrix coordinates corresponding to the target difference contour and the target mapping, wherein the second rectangular box is used to represent the potential position of the difference point in the second image. Then, the detection system determines the target coordinates based on the first rectangular box and the second rectangular box.

[0073] Optionally, the detection system determines a first rectangular frame based on the circumscribed matrix coordinates corresponding to the target difference contour (i.e., the contour formed by the set of points with significant values in the second difference matrix), wherein the circumscribed matrix coordinates refer to the coordinates of the minimum rectangular frame surrounding the target difference contour, and the first rectangular frame contains all the difference points in the contour. The detection system accurately frames the potential change position of the difference contour area in the first image to facilitate subsequent further analysis or image processing of the area, for example, cropping, enlarging or viewing the changed area.

[0074] Optionally, the detection system determines the second rectangular frame based on the circumscribed matrix coordinates corresponding to the target difference contour in combination with the target mapping (i.e., the homography transformation relationship between the two images), wherein the target mapping is obtained through key point detection and matching, and homography transformation extraction, and the process of determining the second rectangular frame is the process of mapping the first rectangular frame from the first image to the second image through homography transformation, thereby determining the potential position of the difference point in the second image. The above process ensures that even if the two images are misaligned, the difference position can be accurately located in the second image.

[0075] In the above-mentioned embodiment, the detection system solves the problem of locating difference points caused by image misalignment by introducing rectangular frame determination and coordinate mapping technology based on difference contours, thereby improving the accuracy and robustness of image change detection. Compared with existing technologies, this method can more accurately identify and locate changed areas in complex scenarios, such as natural landscape changes captured by high-altitude cameras, reducing the probability of false alarms and missed detections, and providing more reliable technical support for the real-time monitoring and protection of natural resources.

[0076] In an optional embodiment, the detection system first segments the first image according to the first rectangular frame to obtain a first sub-image. Thereafter, the detection system segments the second image according to the second rectangular frame to obtain a second sub-image. Then, the detection system obtains the structural similarity between the first sub-image and the second sub-image. Finally, when the structural similarity is greater than a preset threshold, the detection system uses the coordinates of the first rectangular frame and the coordinates of the second rectangular frame as target coordinates.

[0077] Optionally, after determining the first rectangular frame, the detection system segments the first image to obtain the image block within the rectangular frame, i.e., the first sub-image, so that the detection system focuses on the suspected change area determined by the change detection algorithm, thereby narrowing the range of images that need to be processed subsequently and improving the efficiency and accuracy of structural similarity calculation; based on the second rectangular frame, the detection system segments the second image to obtain a second sub-image, wherein the second rectangular frame is converted from the first rectangular frame through target mapping, thereby ensuring that the position corresponding to the difference point in the second image is also accurately framed.

[0078] Optionally, the detection system obtains the structural similarity (Structural Similarity Index, SSIM) between the first sub-image and the second sub-image, wherein the structural similarity comprehensively evaluates the similarity between the first sub-image and the second sub-image based on brightness, contrast and structural information, which can be closer to the human visual system's perception of image differences. If the structural similarity between the first sub-image and the second sub-image is greater than a preset threshold (the smaller the value, the more similar), then the coordinates of the first rectangular box and the second rectangular box are used as target coordinates, indicating that there is indeed a difference between the first sub-image and the second sub-image. When the structural similarity is less than or equal to the preset threshold, it means that the two sub-images are very similar and there is no substantial change. At this time, the area can be excluded as a difference point, further improving the accuracy of difference detection.

[0079] In the above embodiment, the detection system introduces structural similarity calculation in the image comparison process to filter out false positive difference points caused by changes in the posture or focal length of the image acquisition device, ensuring that the detected difference points can truly reflect the changes in the natural scene. This method not only improves the accuracy and reliability of change detection, but also reduces the false alarm rate of the detection system. For natural resource change monitoring, technicians can locate the change areas that really need attention more quickly, thereby improving monitoring efficiency and the timeliness of decision-making.

[0080] In an optional implementation, the detection system first obtains the camera height and camera focal length, wherein the camera height is the height at which the image acquisition device is installed when acquiring the original image, and the camera focal length is the focal length set by the image acquisition device when acquiring the original image. Afterwards, the detection system determines the first parameter and the second parameter based on the camera height and the camera focal length, wherein the first parameter is used to determine the density of the matrix points generated by feature extraction of the image, and the second parameter is the area ratio filtering threshold corresponding to the difference rectangular box, and the difference rectangular box includes the first rectangular box and the second rectangular box.

[0081] Optionally, the detection system first establishes an objective function, wherein the objective function is used to characterize the changing relationship between the camera height, the camera focal length, the first parameter and the second parameter. Thereafter, when the image acquisition device is installed at different heights and set at different focal lengths, the detection system determines the first parameter and the second parameter based on the objective function.

[0082] In the above-mentioned embodiment, different camera heights and camera focal lengths can lead to significant differences in the size, details, and background environment of entities in the images captured by the image acquisition device. The first parameter, namely, the matrix point density, determines the density of the matrix points generated during the image preprocessing and feature extraction stages. Generally, a higher matrix point density can capture more details but increases the complexity of the calculation. The second parameter, namely, the filtering threshold of the area ratio of the difference rectangle, is used to filter out those difference areas with too small an area ratio during the difference detection stage, that is, those caused by noise or non-entity factors. The detection system adjusts the values of the first and second parameters according to different camera heights and camera focal lengths.

[0083] For example, for image acquisition devices installed at higher altitudes, the captured entities are smaller due to the wide shooting range. Therefore, a lower matrix point density (first parameter) is required to avoid overfitting details, and a higher area ratio filtering threshold (second parameter) needs to be set to filter out false positive differences. For image acquisition devices installed at lower altitudes or at a specific focal length, the captured entities are relatively large and the captured details are richer. At this time, a higher matrix point density (first parameter) is required, combined with a lower area ratio filtering threshold (second parameter), to ensure that the detection system can accurately capture image changes and avoid missing smaller difference points.

[0084] In the above embodiment, the detection system dynamically adjusts the key matrix point density and area ratio filtering thresholds in image difference detection based on different camera heights and camera focal lengths, thereby realizing adaptive control of the difference detection algorithm parameters. This strategy improves the robustness and accuracy of image difference detection technology. In particular, when dealing with complex and changeable natural scenes, the detection system can adjust the algorithm parameters according to the actual working environment of the camera, avoiding false detection and missed detection problems caused by fixed parameters; in addition, adaptive parameter control helps to balance the performance and resource consumption of the algorithm, ensuring a dynamic balance between the computing performance and computing efficiency of the detection system in different image acquisition scenarios, thereby improving the efficiency of natural resource monitoring and the accuracy of decision-making.

[0085] From the above content, it can be seen that when detecting image differences, the present application first performs a pre-detection on the two images, that is, the difference detection is performed directly based on the image matrices corresponding to the two images with potential background misalignment. When the pre-detection shows that there is a potential difference between the two images, the present application performs a fine detection on the above two images, that is, first obtains the target mapping, and the target mapping is used to characterize the matching results between the key points included in the two images. Subsequently, the present application performs feature extraction on the two images respectively according to the target mapping, so as to obtain the regional features corresponding to the largest rectangular area matched by the key points in the image. After that, the difference detection is further performed based on the regional features corresponding to the key points in the two images (that is, the first matrix and the second matrix) and the target mapping to obtain the actual position of the difference point in the image.

[0086] It can be seen that the present application achieves the purpose of aligning the contrasting fields of view of two images with background misalignment by introducing target mapping in the fine detection step, thereby overcoming the influence of background misalignment on image difference detection, thereby achieving the technical effect of improving the accuracy of image difference detection, and further solving the technical problem in the prior art of low difference detection accuracy caused by the misalignment of the backgrounds of the two images participating in the detection due to the jitter of the image acquisition device when performing difference detection on two images taken by the same image acquisition device.

[0087] In an optional embodiment, an image difference detection algorithm is also provided. Figure 2 is a flow chart of an optional image difference detection algorithm according to an embodiment of the present application, such as Figure 2As shown, this algorithm is divided into two stages: pre-detection and fine detection, wherein the pre-detection is used to quickly detect and filter out a large number of two images to be detected that have the same posture and no difference; the fine detection is performed on the two images to be detected that have differences detected in the pre-detection stage (the same posture has real differences and different postures are mistakenly detected as having differences), and then fine detection is performed, and the misalignment of the two images to be detected is corrected before difference detection is performed, that is, this algorithm first pre-detects the two images to be detected, and if there are differences, further fine detection is performed, otherwise, the detection is terminated when there are no differences between the two images to be detected.

[0088] In an optional embodiment, an image difference pre-detection algorithm is also provided. Figure 3 is a flow chart of an optional image difference pre-detection algorithm according to an embodiment of the present application, such as Figure 3 As shown, the algorithm includes:

[0089] (1) Image preprocessing: Load the two images to be compared (i.e., two original images) and perform corresponding preprocessing. The preprocessing includes image grayscale processing and image size determination steps. This application only supports two images of the same width and height by default (images of different widths and heights can be scaled to the same size in this application, but this will affect the accuracy of difference detection).

[0090] (2) Generate and calculate SIFT descriptors of matrix points: Generate matrix points according to the width and height corresponding to the two images (the denser the generated matrix points, the greater the subsequent calculation amount and the higher the detection accuracy. However, after the matrix points reach a certain density, the false detection will gradually increase. Therefore, this application reasonably controls the threshold of generating matrix points through the first parameter), and calculate the SIFT descriptors of the matrix points of the two images respectively.

[0091] (3) Matrix point difference analysis: Calculate the feature distance between the matrix point SIFT descriptors corresponding to the two images, filter and obtain the matrix points (i.e., difference points) that exceed the threshold (i.e., the second parameter), and obtain the difference matrix.

[0092] (4) Difference extraction: Scale the difference matrix to the original size of the image to be detected, extract the contours of the difference points, eliminate isolated points and blocks with small contour areas in the process, and obtain the coordinates of the circumscribed rectangle based on the filtered contour blocks. The circumscribed rectangle corresponds to the corresponding position of the original image, which is the pixel point with potential difference.

[0093] (5) Similarity calculation and obtaining difference rectangles: Based on the difference rectangles obtained in the previous step, small images are captured on the first image and the second image respectively. The structural similarity between the corresponding small images is calculated using a preset algorithm (such as the SSIM algorithm) to determine whether there are differences in the positions of the corresponding small images, and the coordinates of the rectangles whose structural similarity exceeds a threshold (the smaller the value, the more similar) are returned.

[0094] If there are differences between the two images, the algorithm will return the coordinates of all the rectangular boxes where the differences exist, and further perform fine detection of image differences.

[0095] In an optional embodiment, a fine image difference detection algorithm is also provided. Figure 4 is a flow chart of an optional image difference fine detection algorithm according to an embodiment of the present application, such as Figure 4 As shown,

[0096] (1) Image preprocessing: Load the two images to be compared (i.e., two original images) and perform corresponding preprocessing. The preprocessing includes image grayscale processing and image size determination steps. This application only supports two images of the same width and height by default (images of different widths and heights can be scaled to the same size in this application, but this will affect the accuracy of difference detection).

[0097] (2) Key point detection: The SIFT algorithm is used to perform key point detection on the two images to be detected.

[0098] (3) Key point matching: perform key point matching on the key points extracted from the two images.

[0099] (4) Extracting homography: Based on the key point matching results obtained in the previous step, extract the homography of the two images respectively, and establish a homography mapping relationship (i.e., target mapping) between the first image and the second image.

[0100] (5) Generate and calculate SIFT descriptors of matrix points: Generate matrix points based on the extracted homography transformation results (different from the pre-detection stage, the range of the matrix generated in this step is not the entire first and second images, but the largest rectangular area matched by the key points in the first and second images), and calculate the SIFT descriptors of the matrix points of the two images respectively.

[0101] (6) Matrix point difference analysis: Calculate the feature distance of the SIFT descriptors of the corresponding matrix points of the two images, filter out the matrix points exceeding the threshold (i.e., difference points) to obtain the difference matrix of the first image.

[0102] (7) Difference extraction: Scale the difference matrix to the original size of the first image and extract the contours of the difference points. In this process, isolated points or blocks with small contour areas need to be eliminated. Based on the filtered contour blocks, the coordinates of the circumscribed rectangle are obtained. The corresponding position of the circumscribed rectangle in the first image is the pixel point where the potential difference exists.

[0103] (8) Similarity calculation and obtaining difference rectangles: Based on the difference rectangle of the first image obtained in the previous step, combined with the previously obtained homography mapping relationship from the first image to the second image, the difference rectangle of the second image can be obtained, and then the first image and the second image are respectively cut into small images. The structural similarity between the corresponding small images is calculated by a preset algorithm to determine whether there is a difference in the position of the corresponding small images, and the coordinates of the rectangles whose structural similarity exceeds the threshold (the smaller the value, the more similar) are returned.

[0104] At this point, if there are differences between the two images, the algorithm will return the coordinates of all the rectangles where the differences occur.

[0105] From the above content, it can be seen that compared with pre-detection, fine detection has additional steps such as image key point detection, key point matching, extraction of homography transformation, and when finally obtaining the difference matrix, it is necessary to obtain the difference rectangle of the second image according to the homography transformation mapping relationship (because there is potential misalignment between the two images, the pixel coordinate position of the difference matrix of the second image is different from the pixel coordinate position of the difference matrix of the first image). This increases the complexity of the algorithm and increases the algorithm execution time to a certain extent in the key point matching link. Relevant test data shows that under the same parameter conditions, the algorithm execution time of fine detection is 5-10 times that of pre-detection.

[0106] Optionally, the preset algorithm can also be set to an ORB (Oriented FAST and Rotated BRIEF) algorithm or a SURF (Speeded Up Robust Features) algorithm.

[0107] Optionally, the above algorithm logic involves a first parameter and a second parameter, wherein the first parameter is used to control the density of the generated matrix points, and the second parameter is used to determine the filtering threshold of the area ratio of the difference rectangular frame. Before performing difference detection, the present application first tests and records different camera heights and camera focal lengths, and selects the parameter value for the best detection result while taking into account the algorithm performance, thereby performing statistical analysis on the best observation data obtained to establish a functional relationship between the camera height, camera focal length and the two parameters, and introduces this function in the above algorithm process to adaptively control the parameter value.

[0108] In summary, the image difference detection algorithm described in the above embodiment has the following functions:

[0109] (1) Improved effectiveness and accuracy of difference extraction: After detecting the difference matrix, the technical solution of the present application extracts difference contours from the difference matrix, obtains the circumscribed rectangle of each extracted contour, intercepts a small image at the original image position where the circumscribed rectangle is located, and calculates the structural similarity of the two small difference images corresponding to the two images. Then, whether there is a difference is further confirmed based on the structural similarity. In the prior art, after detecting the difference matrix, the difference points are clustered by using cluster analysis, and the circle drawn with each cluster center and the corresponding radius is the difference part. There is a problem of cluster uncertainty and overlapping of the drawn difference circles.

[0110] Compared with the common solutions for image difference comparison in the prior art, the present application can avoid the problem of difference extraction overlap through difference contour extraction, and the further difference confirmation mechanism can improve the accuracy of the final difference extraction.

[0111] (2) The algorithm accuracy and performance are taken into account in combination with actual application scenarios: the two images to be detected in the real scene may be taken by the same camera in the same posture, or they may not be taken by the same camera in the same posture, that is, there is a certain probability of misalignment between the two images. In the actual detection process, the proportion of images with real differences is small. Therefore, this application designs two detection stages. The first stage is pre-detection, which is used to quickly filter out a large number of images with no differences in the same posture. The second stage is fine detection, which is used to perform more detailed detection and confirmation on a small number of images with potential differences. The combination of the two modules can take into account both algorithm accuracy and performance.

[0112] (3) Adaptive control of relevant parameters involved in the algorithm: This application obtains best practice data of different thresholds through test experiments in real scenes, and performs statistical analysis on the best practice data to establish a functional relationship between camera height, camera focal length and corresponding parameters, thereby achieving adaptive control of relevant parameters involved in the algorithm, and thus improving the robustness of the entire algorithm.

[0113] According to another aspect of the embodiment of the present application, an image difference detection device is further provided. Figure 5 is a schematic diagram of an optional image difference detection device according to an embodiment of the present application, such as Figure 5 As shown, the image difference detection device includes: a pre-detection unit 501 , a first acquisition unit 502 , a feature extraction unit 503 and a first determination unit 504 .

[0114] Optionally, a pre-detection unit is used to perform pre-detection on the first image and the second image, wherein the pre-detection is at least used to directly perform difference detection based on the image matrix corresponding to the two images; a first acquisition unit is used to obtain a target mapping when the pre-detection result is a preset result, wherein the target mapping is used to characterize the matching result between the first key point and the second key point, the first key point is a pixel point in the first image, and the second key point is a pixel point in the second image, and the preset result is used to characterize the potential difference point between the first image and the second image; a feature extraction unit is used to extract features from the first image and the second image based on the target mapping to obtain a first matrix and a second matrix, wherein the first matrix is used to characterize the regional features of the largest rectangular area matched by the first key point in the first image, and the second matrix is used to characterize the regional features of the largest rectangular area matched by the second key point in the second image; a first determination unit is used to determine the target coordinates based on the first matrix, the second matrix and the target mapping, wherein the target coordinates are used to characterize the actual position of the difference point corresponding to the first image and the second image.

[0115] In an optional embodiment, the image difference detection device further includes: a second acquisition unit and a preprocessing unit.

[0116] Optionally, the second acquisition unit is used to acquire two original images, wherein the two original images are images acquired by the same image acquisition device at different times; the preprocessing unit is used to preprocess the two original images to obtain a first image and a second image, wherein the preprocessing is at least used to perform grayscale processing and size alignment processing on the two original images.

[0117] In an optional embodiment, the pre-detection unit includes: a first feature extraction subunit, a second feature extraction subunit and a pre-detection subunit.

[0118] Optionally, the first feature extraction subunit is used to perform feature extraction on the first image to obtain a first image matrix, wherein the first image matrix is used to characterize the local features of the key points included in the first image; the second feature extraction subunit is used to perform feature extraction on the second image to obtain a second image matrix, wherein the second image matrix is used to characterize the local features of the key points included in the second image; and the pre-detection subunit is used to perform pre-detection on the first image and the second image based on the first image matrix and the second image matrix.

[0119] In an optional embodiment, the first determining unit includes: a first acquiring subunit, an updating subunit, a first determining subunit, and a second determining subunit.

[0120] Optionally, the first acquisition subunit is used to obtain a first difference matrix corresponding to the first matrix and the second matrix, wherein the values in the first difference matrix are used to represent the distance between the SIFT descriptors included in the first matrix and the SIFT descriptors included in the second matrix; the update subunit is used to update the size of the first difference matrix to obtain a second difference matrix, wherein the size of the second difference matrix is equal to the original size of the first image, and the original size of the first image is the size of the first image before preprocessing; the first determination subunit is used to determine the target difference contour based on the second difference matrix, wherein the target difference contour is the contour corresponding to the difference points included in the second difference matrix; the second determination subunit is used to determine the target coordinates based on the target difference contour and the target mapping.

[0121] In an optional embodiment, the second determining subunit includes: a first determining module, a second determining module, and a third determining module.

[0122] Optionally, the first determination module is used to determine a first rectangular box based on the circumscribed matrix coordinates corresponding to the target difference contour, wherein the first rectangular box is used to represent the potential position of the difference point in the first image; the second determination module is used to determine a second rectangular box based on the circumscribed matrix coordinates corresponding to the target difference contour and the target mapping, wherein the second rectangular box is used to represent the potential position of the difference point in the second image; and the third determination module is used to determine the target coordinates based on the first rectangular box and the second rectangular box.

[0123] In an optional embodiment, the third determination module includes: a first segmentation submodule, a second segmentation submodule, an acquisition submodule, and a determination submodule.

[0124] Optionally, the first segmentation submodule is used to segment the first image according to the first rectangular frame to obtain a first sub-image; the second segmentation submodule is used to segment the second image according to the second rectangular frame to obtain a second sub-image; the acquisition submodule is used to obtain the structural similarity between the first sub-image and the second sub-image; and the determination submodule is used to use the coordinates of the first rectangular frame and the coordinates of the second rectangular frame as target coordinates when the structural similarity is greater than a preset threshold.

[0125] In an optional embodiment, the image difference detection device further includes: a second acquiring unit and a second determining unit.

[0126] Optionally, a second acquisition unit is used to obtain a camera height and a camera focal length, wherein the camera height is the height at which the image acquisition device is installed when acquiring the original image, and the camera focal length is the focal length set by the image acquisition device when acquiring the original image; a second determination unit is used to determine a first parameter and a second parameter based on the camera height and the camera focal length, wherein the first parameter is used to determine the density of the matrix points generated by feature extraction of the image, and the second parameter is an area ratio filtering threshold corresponding to the difference rectangular box, and the difference rectangular box includes a first rectangular box and a second rectangular box.

[0127] From the above content, it can be seen that when detecting image differences, the present application first performs a pre-detection on the two images, that is, the difference detection is performed directly based on the image matrices corresponding to the two images with potential background misalignment. When the pre-detection shows that there is a potential difference between the two images, the present application performs a fine detection on the above two images, that is, first obtains the target mapping, and the target mapping is used to characterize the matching results between the key points included in the two images. Subsequently, the present application performs feature extraction on the two images respectively according to the target mapping, so as to obtain the regional features corresponding to the largest rectangular area matched by the key points in the image. After that, the difference detection is further performed based on the regional features corresponding to the key points in the two images (that is, the first matrix and the second matrix) and the target mapping to obtain the actual position of the difference point in the image.

[0128] It can be seen that the present application achieves the purpose of aligning the contrasting fields of view of two images with background misalignment by introducing target mapping in the fine detection step, thereby overcoming the influence of background misalignment on image difference detection, thereby achieving the technical effect of improving the accuracy of image difference detection, and further solving the technical problem in the prior art of low difference detection accuracy caused by the misalignment of the backgrounds of the two images participating in the detection due to the jitter of the image acquisition device when performing difference detection on two images taken by the same image acquisition device.

[0129] According to another aspect of an embodiment of the present application, a computer program product is further provided. The computer program product includes a stored computer program, wherein when the computer program is run, the computer program product is controlled to execute any one of the above-mentioned image difference detection methods.

[0130] According to another aspect of an embodiment of the present application, an electronic device is provided, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any of the above-mentioned image difference detection methods by executing the executable instructions.

[0131] Optionally, Figure 6 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Figure 6As shown, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, any one of the above-mentioned image difference detection methods is implemented.

[0132] The above-mentioned embodiments or examples disclosed in this application are not exhaustive, but are only illustrations of some embodiments or examples, and are not intended to be specific limitations on the scope of protection disclosed in this application. In the absence of contradiction, each step in a certain embodiment or example in this application can be implemented as an independent example, and the steps can be arbitrarily combined. For example, the solution after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be arbitrarily combined; in addition, the various embodiments or examples can be arbitrarily combined. For example, some or all of the steps in different embodiments or examples can be arbitrarily combined, and a certain embodiment or example can be arbitrarily combined with the optional methods or optional examples of other embodiments or examples.

[0133] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0134] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0137] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-permanent storage in a computer-readable medium, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0138] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0139] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0140] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for detecting image differences, characterized in that: include: Performing a pre-detection on the first image and the second image, wherein the pre-detection is at least used to directly perform difference detection based on image matrices corresponding to the two images; When the pre-detection result is a preset result, obtaining a target mapping, wherein the target mapping is used to represent a matching result between a first key point and a second key point, the first key point being a pixel point in the first image, the second key point being a pixel point in the second image, and the preset result being used to represent a potential difference between the first image and the second image; Performing feature extraction on the first image and the second image based on the target mapping to obtain a first matrix and a second matrix, wherein the first matrix is used to represent regional features of the largest rectangular area matched by the first key point in the first image, and the second matrix is used to represent regional features of the largest rectangular area matched by the second key point in the second image; Target coordinates are determined based on the first matrix, the second matrix, and the target mapping, wherein the target coordinates are used to represent actual positions of difference points corresponding to the first image and the second image.

2. The image difference detection method according to claim 1, characterized in that: Before pre-detecting the first image and the second image, the image difference detection method further includes: Acquire two original images, wherein the two original images are images acquired by the same image acquisition device at different times; The two original images are preprocessed to obtain the first image and the second image, wherein the preprocessing is at least used to perform grayscale processing and size alignment processing on the two original images.

3. The image difference detection method according to claim 1, wherein: Pre-detecting the first image and the second image includes: Performing feature extraction on the first image to obtain a first image matrix, wherein the first image matrix is used to represent local features of key points included in the first image; Performing feature extraction on the second image to obtain a second image matrix, wherein the second image matrix is used to represent local features of key points included in the second image; The pre-detection is performed on the first image and the second image according to the first image matrix and the second image matrix.

4. The image difference detection method according to claim 1, wherein: Determining target coordinates according to the first matrix, the second matrix, and the target mapping includes: Obtaining a first difference matrix corresponding to the first matrix and the second matrix, wherein a value in the first difference matrix is used to represent a distance between a SIFT descriptor included in the first matrix and a SIFT descriptor included in the second matrix; updating the size of the first difference matrix to obtain a second difference matrix, wherein the size of the second difference matrix is equal to the original size of the first image, and the original size of the first image is the size of the first image before preprocessing; determining a target difference profile according to the second difference matrix, wherein the target difference profile is a profile corresponding to the difference points included in the second difference matrix; The target coordinates are determined based on the target difference profile and the target map.

5. The image difference detection method according to claim 4, characterized in that: Determining the target coordinates based on the target difference profile and the target mapping includes: determining a first rectangular frame according to the circumscribed matrix coordinates corresponding to the target difference contour, wherein the first rectangular frame is used to represent a potential position of the difference point in the first image; determining a second rectangular frame according to the circumscribed matrix coordinates corresponding to the target difference contour and the target mapping, wherein the second rectangular frame is used to represent a potential position of the difference point in the second image; The target coordinates are determined according to the first rectangular frame and the second rectangular frame.

6. The image difference detection method according to claim 5, characterized in that: Determining the target coordinates according to the first rectangular frame and the second rectangular frame includes: Segmenting the first image according to the first rectangular frame to obtain a first sub-image; Segmenting the second image according to the second rectangular frame to obtain a second sub-image; obtaining structural similarity between the first sub-image and the second sub-image; When the structural similarity is greater than a preset threshold, the coordinates of the first rectangular frame and the coordinates of the second rectangular frame are used as the target coordinates.

7. The image difference detection method according to claim 3, characterized in that: Before performing feature extraction on the first image, the image difference detection method further includes: Obtaining a camera height and a camera focal length, wherein the camera height is the height at which the image acquisition device is installed when acquiring the original image, and the camera focal length is the focal length set by the image acquisition device when acquiring the original image; The first parameter and the second parameter are determined based on the camera height and the camera focal length, wherein the first parameter is used to determine the density of the matrix points generated by feature extraction of the image, and the second parameter is the area ratio filtering threshold corresponding to the difference rectangular box, and the difference rectangular box includes the first rectangular box and the second rectangular box.

8. An image difference detection device, characterized in that: include: a pre-detection unit, configured to perform pre-detection on the first image and the second image, wherein the pre-detection is at least configured to directly perform difference detection based on image matrices corresponding to the two images; a first acquisition unit, configured to acquire a target mapping when the pre-detection result is a preset result, wherein the target mapping is used to represent a matching result between a first key point and a second key point, the first key point being a pixel point in the first image, the second key point being a pixel point in the second image, and the preset result being used to represent a potential difference between the first image and the second image; a feature extraction unit, configured to perform feature extraction on the first image and the second image based on the target mapping to obtain a first matrix and a second matrix, wherein the first matrix is used to represent regional features of the largest rectangular area matched by the first key point in the first image, and the second matrix is used to represent regional features of the largest rectangular area matched by the second key point in the second image; A first determining unit is configured to determine target coordinates based on the first matrix, the second matrix, and the target mapping, wherein the target coordinates are used to represent actual positions of difference points corresponding to the first image and the second image.

9. A computer program product, characterized in that The computer program product comprises a computer program, wherein when the computer program is run, the computer program product is controlled to execute the image difference detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The system comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image difference detection method according to any one of claims 1 to 7.

Images