A method for intelligently detecting and filtering differences in specific areas of images based on deep learning

Through intelligent detection methods based on deep learning, combined with deep learning models and morphological operations, the recognition accuracy and efficiency of image difference detection in complex scenarios is solved, and efficient detection of specific regions is achieved, reducing the false alarm rate .

CN118570568BActive Publication Date: 2024-12-06LUCULENT SMART TECHNOLOGIES CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411053658.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-12-06
Estimated Expiration
2044-08-02

AI Technical Summary

Technical Problem

The existing image difference detection methods are difficult to adapt to changing environments, and combine complex scenes and changing environments to analyze and identify them, improve recognition accuracy and efficiency, and reduce false alarm rates.

Method used

Using intelligent detection methods based on deep learning, the differential images are initially screened and filtered extracted through the deep learning detection model, image alignment, difference enhancement and difference suppression are performed, and the difference detection model is trained in a specific region with the adaptive weight. Functional and morphological operations to achieve accurate detection of the degree of difference in specific regions.

Benefits of technology

It improves the recognition accuracy and efficiency of differential detection in specific areas, reduces the false alarm rate, enhances the robustness and adaptability of differential detection of images, and ensures efficient recognition performance in complex scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118570568B_ABST
    Figure CN118570568B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for intelligently detecting and filtering differences in specific regions of images based on deep learning, and relates to the field of image processing technology, including preliminarily screening difference images and extracting objects to be filtered through a deep learning detection model; aligning graphics without differences and graphics with differences, calculating the differences in the registered grayscale images and performing difference enhancement; traversing the contours of the difference images, obtaining the coordinates of the upper left corner of the contours and the width and height, and performing difference suppression in sequence. The method for intelligently detecting and filtering differences in specific regions of images based on deep learning provided by the present invention is combined with a deep learning model to improve the accuracy of difference detection in specific regions and the recognition of parts that need to be filtered. The images are geometrically aligned through perspective transformation to ensure that the corresponding relationship between the two images is consistent when compared, thereby improving the accuracy of difference detection. Grayscale conversion is performed to reduce data dimensions and computational complexity, while improving the robustness of subsequent feature detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for intelligently detecting and filtering differences in specific areas of images based on deep learning. Background Art

[0002] With the development of computer vision technology, image difference detection methods have been widely used in many fields, including industrial inspection, medical imaging, security monitoring, and autonomous driving. Traditional image processing algorithms, such as template matching, histogram comparison, optical flow, feature point matching, etc., have been widely used and have achieved remarkable results in many applications. With the rise of deep learning technology, image processing methods based on convolutional neural networks (CNNs) have shown significant advantages in recognition accuracy and processing speed. Deep learning models can automatically extract high-level features from images, adapt to complex scenes and changing environments, and greatly improve the performance of image difference detection.

[0003] Existing image difference detection methods are usually based on traditional image processing algorithms, such as template matching, histogram comparison, etc. These methods have limited effects when dealing with complex scenes, dynamic changes and noise interference, such as specific area detection, initial screening of differences, targeted filtering, lighting changes, etc. At the same time, with the development of deep learning technology, image processing methods based on deep learning have significant advantages in recognition accuracy and processing speed, but in practical applications, especially in industrial scene channel occupancy recognition, how to combine complex scenes, changing environments, special needs, etc. to improve recognition accuracy and efficiency and reduce false alarm rates, there are still challenges. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is that the existing image difference detection method is difficult to adapt to the changing environment, analyze and identify in combination with the environment, and how to improve the recognition accuracy and efficiency and reduce the false alarm rate.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for intelligently detecting and filtering the difference of a specific area of ​​a picture based on deep learning, comprising preliminarily screening the difference picture and extracting the object to be filtered through a deep learning detection model. Aligning the non-difference and difference graphics, calculating the difference of the registered grayscale image and performing difference enhancement. Traversing the difference image contour, obtaining the coordinates of the upper left corner of the contour and the width and height, and performing difference suppression in turn.

[0007] As a preferred solution of the method for filtering the difference of specific areas of images based on deep learning intelligent detection described in the present invention, the initial screening of difference images by the deep learning detection model includes collecting no-difference and difference image materials in specific areas, marking the materials, marking whether there are differences, using YOLOv5 to train the marked material set, and outputting a specific area difference detection model. , expressed as:

[0008] ,

[0009] ,

[0010] ,

[0011] ,

[0012] in, is the boundary loss, is the boundary loss weight, is the classification loss, is the classification loss weight, is the confidence loss, is the confidence loss weight, is the resulting bounding box, is the input bounding box, is the probability that the input category label is the same as the model prediction label, is the true confidence of the input and is the model prediction confidence, represents the number of training samples used to calculate the classification loss, represents the number of training samples used to calculate the confidence loss, is the number of labeled categories.

[0013] Use the specific area difference detection model produced by training to identify the picture, and initially screen whether there is a difference. After judgment, obtain the specific area image without difference and specific area images with differences .

[0014] As a preferred solution of the method for intelligently detecting the difference of specific areas of filtered images based on deep learning described in the present invention, wherein: the extraction of the objects to be filtered includes collecting the image materials of the objects to be filtered according to the image filtering requirements, marking the objects to be filtered of the material images, training the marked material set, determining parameters and weights, inputting the specific area difference detection model for weight replacement to output the object to be filtered detection model, and the object to be filtered detection model uses the model to Identify and determine whether there is an object to be filtered. If so, extract the location information of the object to be filtered.

[0015] As a preferred solution of the method for intelligently detecting and filtering the difference of a specific area of ​​a picture based on deep learning described in the present invention, wherein: aligning the graphics without difference and the graphics with difference includes aligning the images and Convert to grayscale image and , for grayscale images and Perform key point detection and descriptor calculation to obtain feature points and , and feature point descriptors and , add direction information to make the feature points rotation-invariant, and the direction information is expressed as:

[0016] Calculate horizontal gradient and vertical gradient , expressed as:

[0017] ,

[0018] in, is the input grayscale image, For any pixel, for exist The gradient in direction, for exist Gradient in direction, calculate gradient magnitude and the gradient direction :

[0019] ,

[0020] in, for The gradient amplitude, for The gradient direction defines an adaptive weight function based on local contrast and texture complexity , expressed as:

[0021] ,

[0022] in, Pixel The adaptive weight of is the feature point to be added to the direction, is the standard deviation of the Gaussian distribution;

[0023] The weighted gradient direction histogram is expressed as:

[0024] ,

[0025] Angle The weighted gradient direction histogram value of patch is The neighborhood area;

[0026] is the Dirac delta function, when equal The value is 1 when , otherwise it is 0;

[0027] Get histogram The angle corresponding to the maximum value of The main direction :

[0028] ,

[0029] Descriptors of feature points in two images and Perform matching, calculate the distance between each pair of feature point descriptors, and obtain the matching result M mr ;

[0030] Calculate the best homography matrix from the matching result M mr Randomly select 4 pairs of matching feature points and calculate the initial homography matrix. and For grayscale images and The corresponding feature point coordinates in the homography matrix H are expressed as:

[0031] ,

[0032] in, Indicates that the homography matrix H acts on the grayscale image The feature point coordinates The new feature point position obtained later;

[0033] Calculate the projection error of all matching feature points after transformation by the homography matrix H , the error is defined as the Euclidean distance between the transformed point and the actual point:

[0034] ,

[0035] Dynamically adjust the inlier threshold according to the median of the error distribution;

[0036] ,

[0037] in, is the interior point adjustment coefficient, and its value range is [1,2];

[0038] Determine whether the projection error is less than the set threshold , if it is less than, the matching feature point is considered to be an interior point;

[0039] Record the number of inliers, repeat the steps, select the homography matrix with the largest number of inliers as the best homography matrix, and transform the grayscale image according to the best homography matrix. Perform a perspective transformation, The image coordinate system is transformed to The image coordinate system is aligned in space to achieve image registration and obtain the registered grayscale image .

[0040] As a preferred solution of the method for intelligently detecting and filtering the difference of a specific area of ​​a picture based on deep learning described in the present invention, wherein: the calculation of the difference of the grayscale image after registration and the difference enhancement include: and Perform mean filtering and denoising to obtain the denoised image and , calculate the denoised image and The difference image Diff between the two images, each pixel value of the difference image represents the difference in pixel values ​​between the two images.

[0041] The difference image is binarized to highlight the difference.

[0042] Morphological operations are applied to enhance the continuity of the difference regions. Closing operations are used to first dilate and then erode to fill small holes and connect adjacent difference regions to obtain contours in the difference image.

[0043] As a preferred solution of the method for filtering the difference of a specific area of ​​an image based on deep learning intelligent detection described in the present invention, wherein: the traversal of the difference image contour, obtaining the coordinates of the upper left corner and the width and height of the contour, and sequentially performing difference suppression include suppressing the size of the difference contour according to a preset image size threshold T, and suppressing contours smaller than the specified size by calculating the contour area and comparing it with the preset image size threshold T, which is expressed as:

[0044] ,

[0045] in Indicates the suppression flag, 1 means suppression is required;

[0046] The edge of the difference contour is suppressed according to the preset boundary calculation threshold M, and the difference contour in the edge area is removed. The difference contour must meet the restriction condition to be considered in the valid area;

[0047] Restrictions include:

[0048] The left edge of the difference contour does not exceed the left edge limit, ;

[0049] The upper edge of the difference contour does not exceed the upper edge limit, ;

[0050] The right edge of the difference contour does not exceed the right edge limit, ;

[0051] The lower edge of the difference contour does not exceed the lower edge limit, ;

[0052] in, is the coordinate of the upper left corner of the difference contour, is the width and height of the difference contour, For specific area images with differences The width and height of .

[0053] As a preferred solution of the method for filtering the difference of a specific area of ​​an image based on deep learning intelligent detection described in the present invention, wherein: the traversal of the difference image contour, obtaining the coordinates of the upper left corner and the width and height of the contour, and sequentially performing difference suppression also includes using an adaptive difference threshold algorithm to calculate a global threshold, assuming that the original image after denoising is , the difference image is , and the global mean and standard deviation are and :

[0054] ,

[0055] ,

[0056] in, Represents the original image after denoising The gray value of the i-th pixel in , N is the total number of pixels in the image;

[0057] Using Difference Images Calculate the global difference:

[0058] ,

[0059] in, Represents the difference image after denoising The gray value of the i-th pixel in;

[0060] Dynamically adjust the difference threshold according to the global features, assuming that the initial threshold is , the adjusted global threshold is It is expressed as:

[0061] ,

[0062] in, is the global threshold adjustment coefficient;

[0063] Use the adaptive difference threshold algorithm to calculate the local threshold, create a blank mask of the same size as the difference image Diff, and draw the current contour area on the mask;

[0064] Use a mask to shield the non-difference contour area in the difference image Diff, set the pixel value of the non-difference area to 0, and only retain the pixel value of the difference contour area. Accumulate the non-zero pixel values ​​in the difference contour area and calculate the average to obtain the difference average value in the difference contour area;

[0065] Compare with the difference threshold D. If it is not greater than D, it means that the difference does not meet expectations and needs to be suppressed.

[0066] Traverse the suppressed difference contours, calculate the difference coincidence with the position information of the object to be filtered obtained by the initial screening of the difference image through the deep learning detection model, and determine whether the object to be filtered needs to be filtered according to the preset coincidence threshold C;

[0067] The single difference coincidence calculation includes:

[0068] Get the difference profile Upper left corner coordinates And the lower right corner coordinates , need to filter Upper left corner coordinates And the lower right corner coordinates ;

[0069] By judging Are the upper left corner coordinates and lower right corner coordinates in Within the range, determine Is it completely included in Inside, it is expressed as:

[0070] ,

[0071] calculate and The overlapped area, the coordinates of the upper left corner of the overlapped area:

[0072] ,

[0073] The coordinates of the lower right corner of the overlapping area:

[0074] ,

[0075] Calculate the area of ​​the overlapping area. If the width or height of the overlapping area is 0, it means there is no overlapping area.

[0076] ,

[0077] Otherwise, calculate the area of ​​the overlapping region, expressed as:

[0078] ,

[0079] Calculate the overlap, which is the ratio of the overlap area to the The ratio of the areas is expressed as:

[0080] ,

[0081] If they overlap and the overlap ratio is greater than the overlap threshold C, it means that the current difference contour is to be filtered, and filtering is performed. If there is still a difference contour after difference suppression, the contour information is drawn in the specific area image where the difference exists. The required image difference of a specific area is obtained.

[0082] Another object of the present invention is to provide a system for intelligently detecting and filtering the difference of specific areas of images based on deep learning, which can dynamically filter and abandon some areas that do not meet the conditions through deep learning detection results, thereby improving detection efficiency and accuracy. This solves the problem that the current image difference detection method has low efficiency and insufficient accuracy.

[0083] As a preferred solution of the system for filtering the difference of specific areas of images based on deep learning intelligent detection described in the present invention, it includes: a preliminary screening module, an alignment module, and a difference determination module. The preliminary screening module is used to preliminarily screen the difference images and extract the objects to be filtered through the deep learning detection model. The alignment module is used to align the graphics without difference and the graphics with difference, calculate the difference of the grayscale images after registration and perform difference enhancement. The difference determination module is used to traverse the contour of the difference image, obtain the coordinates of the upper left corner of the contour and the width and height, and perform difference suppression in turn.

[0084] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a method for intelligently detecting and filtering differences in specific areas of a picture based on deep learning.

[0085] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for intelligently detecting and filtering differences in specific areas of a picture based on deep learning.

[0086] Beneficial effects of the present invention: The method for intelligently detecting the difference of specific areas of filtered images based on deep learning provided by the present invention is combined with a deep learning model to improve the accuracy of detecting differences in specific areas and identifying the parts that need to be filtered. The images are geometrically aligned through perspective transformation to ensure that the correspondence between the two images is consistent when compared, thereby improving the accuracy of difference detection. Grayscale conversion is performed to reduce data dimensions and computational complexity, while improving the robustness of subsequent feature detection. The present invention achieves better results in terms of efficiency, computational accuracy, and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0088] Figure 1 An overall flow chart of a method for intelligently detecting and filtering differences in specific areas of images based on deep learning provided in the first embodiment of the present invention.

[0089] Figure 2 A flowchart of image specific area difference detection and filtering method based on deep learning intelligent detection and filtering of image specific area difference is provided in the first embodiment of the present invention.

[0090] Figure 3 A method for intelligently detecting and filtering the difference of specific areas of images based on deep learning provided in the second embodiment of the present invention is provided, which locks the non-occupied image of the specific area image that needs to be compared for image difference after detection by a deep learning model.

[0091] Figure 4 A method for intelligently detecting and filtering the difference of specific areas of a picture based on deep learning provided in the second embodiment of the present invention is provided. After detection by a deep learning model, an occupied image of an image of a specific area for which image difference comparison is required is locked.

[0092] Figure 5 This is a diagram showing the effect of geometric alignment of a method for intelligently detecting differences in specific areas of filtered images based on deep learning, provided in the second embodiment of the present invention.

[0093] Figure 6A method for intelligently detecting and filtering differences in specific areas of images based on deep learning provided by the second embodiment of the present invention Figure 5 and Figure 4 Difference image after denoising and difference calculation.

[0094] Figure 7 A method for intelligently detecting and filtering differences in specific areas of images based on deep learning provided by the second embodiment of the present invention Figure 6 The difference contour is obtained after difference enhancement, but the result image is not obtained after difference suppression.

[0095] Figure 8 A final effect diagram after suppressing the difference of the difference contours of a method for intelligently detecting the difference of specific areas of filtered images based on deep learning provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0096] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0097] Example 1

[0098] Reference Figure 1-Figure 2 , as an embodiment of the present invention, provides a method for intelligently detecting and filtering differences in specific areas of images based on deep learning, comprising:

[0099] S1: Use the deep learning detection model to preliminarily screen the difference images and extract the objects to be filtered.

[0100] Furthermore, the deep learning detection model is used to initially screen the difference images, including collecting the image materials with no difference and difference in the specific area, marking the materials, and using annotation tools (such as LabelImg, VGG ImageAnnotator) to annotate the specific area of ​​each image to mark whether there is a difference label, and use YOLOv5 to train the marked material set to produce a specific area difference detection model. , expressed as:

[0101] ,

[0102] ,

[0103] ,

[0104] ,

[0105] in, is the boundary loss, is the boundary loss weight, is the classification loss, is the classification loss weight, is the confidence loss, is the confidence loss weight, is the resulting bounding box, is the input bounding box, is the probability that the input category label is the same as the model prediction label, is the true confidence of the input and is the model prediction confidence, represents the number of training samples used to calculate the classification loss, represents the number of training samples used to calculate the confidence loss, is the number of labeled categories. The specific area difference detection model produced by training is used to identify the image, and the initial screening determines whether there is a difference. After the judgment, the specific area image without difference is obtained. and specific area images with differences .

[0106] It should be noted that extracting the objects to be filtered includes collecting the image materials of the objects to be filtered according to the image filtering requirements, marking the objects to be filtered in the material images, training the marked material set, determining parameters and weights, inputting the specific area difference detection model to replace the weights to output the object to be filtered detection model, and the object to be filtered detection model uses the model to Identify and determine whether there is an object to be filtered. If so, extract the location information of the object to be filtered.

[0107] S2: Align the images without difference and with difference, calculate the difference of the grayscale images after registration and perform difference enhancement.

[0108] Furthermore, aligning the indifferent and different images involves and Convert to grayscale image and , for grayscale images and Perform key point detection and descriptor calculation to obtain feature points and , and feature point descriptors and , add direction information to make the feature points rotation-invariant, and the direction information is expressed as:

[0109] Calculate horizontal gradient and vertical gradient , expressed as:

[0110] ,

[0111] in, is the input grayscale image, For any pixel, for exist The gradient in direction, for exist Gradient in direction, calculate gradient magnitude and the gradient direction :

[0112] ,

[0113] in, for The gradient amplitude, for The gradient direction defines an adaptive weight function based on local contrast and texture complexity , expressed as:

[0114] ,

[0115] in, Pixel The adaptive weight of is the feature point to be added to the direction, is the standard deviation of the Gaussian distribution;

[0116] The weighted gradient direction histogram is expressed as:

[0117] ,

[0118] Angle The weighted gradient direction histogram value of patch is The neighborhood area;

[0119] is the Dirac delta function, when equal The value is 1 when , otherwise it is 0;

[0120] Get histogram The angle corresponding to the maximum value of The main direction :

[0121] ,

[0122] Descriptors of feature points in two images and Perform matching, calculate the distance between each pair of feature point descriptors, and obtain the matching result M mr ;

[0123] Calculate the best homography matrix from the matching result M mr Randomly select 4 pairs of matching feature points and calculate the initial homography matrix. and For grayscale images and The corresponding feature point coordinates in the homography matrix H are expressed as:

[0124] ,

[0125] in, Indicates that the homography matrix H acts on the grayscale image The feature point coordinates The new feature point position obtained later;

[0126] Calculate the projection error of all matching feature points after transformation by the homography matrix H , the error is defined as the Euclidean distance between the transformed point and the actual point:

[0127] ,

[0128] Dynamically adjust the inlier threshold according to the median of the error distribution;

[0129] ,

[0130] in, is the interior point adjustment coefficient, and its value range is [1,2];

[0131] Determine whether the projection error is less than the set threshold , if it is less than, the matching feature point is considered to be an interior point;

[0132] Record the number of inliers, repeat the steps, select the homography matrix with the largest number of inliers as the best homography matrix, and transform the grayscale image according to the best homography matrix. Perform a perspective transformation, The image coordinate system is transformed to The image coordinate system is aligned in space to achieve image registration and obtain the registered grayscale image .

[0133] It should be noted that calculating the difference of the registered grayscale image and performing difference enhancement includes and Perform mean filtering and denoising to obtain the denoised image and , calculate the denoised image and The difference image Diff between the two images, each pixel value of the difference image represents the difference in pixel values ​​between the two images.

[0134] The difference image is binarized to highlight the difference.

[0135] Morphological operations are applied to enhance the continuity of the difference regions. Closing operations are used to first dilate and then erode to fill small holes and connect adjacent difference regions to obtain contours in the difference image.

[0136] S3: traverse the difference image contour, obtain the coordinates of the upper left corner and the width and height of the contour, and perform difference suppression in sequence.

[0137] Furthermore, this step actually includes two steps of difference suppression, namely, difference suppression and filter filtering. The difference image contour is traversed to obtain the coordinates of the upper left corner and the width and height of the contour. The difference suppression is performed in sequence, including suppressing the size of the difference contour according to the preset image size threshold T. By calculating the contour area and comparing it with the preset image size threshold T, the contour smaller than the specified size is suppressed, which is expressed as:

[0138] ,

[0139] in Indicates the suppression mark, 1 indicates that suppression is required. The determination of the size threshold depends on the size of the target detection object and the image resolution. In order to filter out too small noise and too large irrelevant areas, the present invention sets a reasonable size threshold range. According to experimental adjustment, the general size threshold range can be set between 1000 and 3000.

[0140] If the image resolution is high and the size of the target object is relatively large, the size threshold can be appropriately increased and set to between 2000 and 3000 to filter out small noise.

[0141] If the target object itself is small, you can lower the size threshold and set the threshold between 1000 and 2000 to ensure that smaller targets can be detected.

[0142] The difference contour is edge suppressed according to the preset boundary calculation threshold M, and the difference contour in the edge area is removed. The difference contour must meet the restriction condition to be considered in the valid area.

[0143] Restrictions include:

[0144] The left edge of the difference contour does not exceed the left edge limit, ;

[0145] The upper edge of the difference contour does not exceed the upper edge limit, ;

[0146] The right edge of the difference contour does not exceed the right edge limit, ;

[0147] The lower edge of the difference contour does not exceed the lower edge limit, ;

[0148] in, is the coordinate of the upper left corner of the difference contour, is the width and height of the difference contour, For specific area images with differences The boundary threshold is used to ensure that the detected object is not too close to the edge of the image, thereby avoiding false detection due to edge effects. The setting of the boundary threshold needs to consider the size of the image and the actual position of the target object. Usually, the boundary threshold can be set between 2% and 5% of the image width or height.

[0149] If the detection target objects are often close to the edge of the image, the boundary threshold can be appropriately lowered and set to between 2% and 3% to ensure that these targets are not missed.

[0150] If the target object is usually located in the center of the image, the boundary threshold can be increased to 4% to 5% to reduce the impact of edge effects on the detection results.

[0151] It should be noted that traversing the difference image contour, obtaining the coordinates of the upper left corner of the contour and the width and height, and sequentially performing difference suppression also includes using an adaptive difference threshold algorithm to calculate the global threshold. Suppose the original image after denoising is , the difference image is , and the global mean and standard deviation are and :

[0152] ,

[0153] ,

[0154] in, Represents the original image after denoising The gray value of the i-th pixel in , N is the total number of pixels in the image;

[0155] Using Difference Images Calculate the global difference:

[0156] ,

[0157] in, Represents the difference image after denoising The gray value of the i-th pixel in;

[0158] Dynamically adjust the difference threshold according to the global features, assuming that the initial threshold is , the adjusted global threshold is It is expressed as:

[0159] ,

[0160] in, is the global threshold adjustment factor.

[0161] Use the adaptive difference threshold algorithm to calculate the local threshold, create a blank mask of the same size as the difference image Diff, draw the current contour area on the mask, use the mask to shield the non-difference contour area in the difference image Diff, set the pixel value of the non-difference area to 0, and only retain the pixel value of the difference contour area. After accumulating the non-0 pixel values ​​in the difference contour area, calculate the average value to obtain the difference average value in the difference contour area. The adaptive difference threshold algorithm here can calculate the global threshold or the local threshold. The calculation method is similar, except that the local threshold divides the image into M*M small blocks, and each block is subjected to the adaptive difference threshold algorithm to obtain the corresponding local threshold.

[0162] According to the actual situation, the global threshold or the local threshold of the current difference profile can be used as the difference threshold D for subsequent difference suppression.

[0163] Compare it with the difference threshold D. If it is not greater than D, it means that the difference does not meet expectations and needs to be suppressed.

[0164] The difference contours after traversal suppression are sequentially compared with the position information of the object to be filtered obtained by the initial screening of the difference image through the deep learning detection model, and the difference coincidence is calculated, and whether the object to be filtered needs to be filtered is determined according to the preset coincidence threshold C.

[0165] The single difference coincidence calculation includes:

[0166] Get the difference profile Upper left corner coordinates And the lower right corner coordinates , need to filter Upper left corner coordinates And the lower right corner coordinates .

[0167] By judging Are the upper left corner coordinates and lower right corner coordinates in Within the range, determine Is it completely included in Inside, it is expressed as:

[0168] ,

[0169] calculate and The overlapped area, the coordinates of the upper left corner of the overlapped area:

[0170] ,

[0171] The coordinates of the lower right corner of the overlapping area:

[0172] ,

[0173] Calculate the area of ​​the overlapping area. If the width or height of the overlapping area is 0, it means there is no overlapping area.

[0174] ,

[0175] Otherwise, calculate the area of ​​the overlapping region, expressed as:

[0176] ,

[0177] Calculate the overlap, which is the ratio of the overlap area to the The ratio of the areas is expressed as:

[0178] ,

[0179] If the two target areas overlap and the overlap ratio is greater than the overlap threshold C, it means that the current difference profile is to be filtered and filtered. The overlap threshold is used to determine the overlap degree of the two target areas. In order to filter out repeated detection and overlapping areas, the present invention sets a reasonable overlap threshold range. Usually, the overlap threshold can be set between 0.3 and 0.7.

[0180] If there is a lot of overlap in the target areas, the overlap threshold can be appropriately increased to between 0.5 and 0.7 to reduce duplicate detection.

[0181] If the target area overlaps less, the overlap threshold can be lowered to between 0.3 and 0.5 to ensure that all targets are detected.

[0182] If there are still difference contours after difference suppression, the contour information is drawn in the specific area image where the difference exists. The required image difference of a specific area is obtained.

[0183] Example 2

[0184] Reference Figure 3-Figure 8 An embodiment of the present invention provides a method for intelligently detecting and filtering the difference of specific areas of an image based on deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0185] First, a detailed description of channel occupancy detection for a certain industrial unit is given.

[0186] Use the images collected by the channel monitoring camera to label the occupied area to be detected and whether it is occupied. Train the channel occupancy detection model to identify the images collected by the camera to obtain the non-occupied images and the occupied images to be compared, such as Figure 3 and Figure 4 shown.

[0187] According to actual business needs, people often pass through and stay briefly in this channel, which is not a channel occupancy situation, so people need to be filtered. The images collected by the camera are recognized in parallel through the personnel detection model to obtain the position information of the people in the occupied image to be compared, such as Figure 4 If no person is detected, the location information that needs to be filtered will not be recorded.

[0188] Considering the special situation that the surveillance camera or other image acquisition devices may have lens shaking, the images to be compared are then geometrically aligned, such as Figure 5 shown.

[0189] After registration, the image is denoised and the difference is calculated to obtain a difference image, such as Figure 6 As shown, the difference image is then enhanced to obtain the difference contour. If the difference is not suppressed, then Figure 7 As shown, there are errors such as size and edges.

[0190] The difference contours are suppressed according to the preset size, boundary, and difference thresholds to control the difference contours to be not too large and not in the edge area (incomplete), and to ignore erroneous differences such as sunlight. If a person is detected, the remaining difference contours after suppression are filtered for people, and the obtained person location information is obtained and the overlap is calculated with the difference contours. The preset overlap threshold is used to determine whether the current difference contour needs to be filtered as a person.

[0191] Draw the difference contours remaining after suppression (filtering) in the occupancy image to be compared, such as Figure 8 As shown, it is then determined whether there is occupancy.

[0192] Table 1 shows the data of the unit using relevant channel occupancy identification methods on the market. The comparison shows that the method of the present invention has more advantages in accuracy and efficiency.

[0193] Table 1 Comparison of channel occupancy identification methods

[0194]

[0195] Example 3

[0196] An embodiment of the present invention provides a system for intelligently detecting and filtering differences in specific areas of images based on deep learning, including a preliminary screening module, an alignment module, and a difference determination module.

[0197] Among them, the preliminary screening module is used to preliminarily screen the difference images through the deep learning detection model and extract the objects to be filtered. The alignment module is used to align the images with and without differences, calculate the difference of the grayscale images after registration, and perform difference enhancement. The difference determination module is used to traverse the difference image contour, obtain the coordinates of the upper left corner of the contour and the width and height, and perform difference suppression in turn.

[0198] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0199] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0200] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk case (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0201] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logical function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0202] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for intelligently detecting and filtering differences in specific areas of images based on deep learning, characterized in that: include: Use the deep learning detection model to preliminarily screen the difference images and extract the objects to be filtered; Align the images without difference and images with difference, calculate the difference of the registered grayscale images and perform difference enhancement; Traverse the difference image contour, obtain the coordinates of the upper left corner and the width and height of the contour, and perform difference suppression in sequence; The alignment of the indifferent and different graphics includes converting the images I1 and I2 into grayscale images G1 and G2, performing key point detection and descriptor calculation on the grayscale images G1 and G2, and obtaining feature points p i and p′ i , and feature point descriptor D i and D′ i , add direction information to make the feature points rotation-invariant, and the direction information is expressed as: Calculate the horizontal gradient G x and the vertical gradient G y , expressed as: Among them, G is the input grayscale image, p is any pixel point, G x (p) is the gradient of p in the x direction, G y (p) is the gradient of p in the y direction, and the gradient magnitude M(p) and gradient direction φ(p) are calculated: Among them, M(p) is the gradient amplitude of p, φ(p) is the gradient direction of p, and an adaptive weight function w(p) based on local contrast and texture complexity is defined, which is expressed as: Among them, w(p) is the adaptive weight of pixel p, p k is the feature point in the direction to be added, σ is the standard deviation of the Gaussian distribution; The weighted gradient direction histogram is expressed as: H(θ) is the weighted gradient direction histogram value of angle θ, and patch is p k The neighborhood area; δ(φ(p)-θ) is the Dirac delta function, which takes the value of 1 when φ(p) equals θ and 0 otherwise; Take the angle corresponding to the maximum value of the histogram H(θ) as p k Main direction: Descriptors D of feature points in two images i and D′ i Perform matching, calculate the distance between each pair of feature point descriptors, and obtain the matching result M mr ; Calculate the best homography matrix from the matching result M mr Randomly select 4 pairs of matching feature points and calculate the initial homography matrix. Let p i =(x i ,y i ) and p′ i =(x′ i , y′ i ) are the coordinates of the corresponding feature points in the grayscale images G1 and G2, and the homography matrix H is expressed as: Among them, Hp i Represents the feature point coordinates p acting on the grayscale image G1 through the homography matrix H i The new feature point position obtained later; Calculate the projection error e of all matching feature points after transformation by the homography matrix H i , the error is defined as the Euclidean distance between the transformed point and the actual point: e i =||Hp i -p′ i || Dynamically adjust the inlier threshold according to the median of the error distribution; τ=α·median(||Hp i -p′ i ||) Among them, α is the interior point adjustment coefficient, and its value range is [1,2]; Determine whether the projection error is less than the set threshold τ. If so, the matching feature point is considered to be an inner point. Record the number of inliers, repeat the steps, select the homography matrix with the largest number of inliers as the best homography matrix, perform perspective transformation on the grayscale image G1 according to the best homography matrix, transform the G1 image coordinate system to the G2 image coordinate system, align them in space, realize image registration, and obtain the registered grayscale image G′1; The calculating the difference of the registered grayscale images and performing difference enhancement comprises performing mean filtering denoising processing on the grayscale images G′1 and G2 to obtain denoised images B1 and B2, and calculating a difference image Diff between the denoised images B1 and B2, wherein each pixel value of the difference image represents a difference in pixel values ​​between the two images; Binarize the difference image to highlight the difference; Morphological operations are applied to enhance the continuity of the difference regions. Closing operations are used to first dilate and then erode to fill small holes and connect adjacent difference regions to obtain contours in the difference image. The traversal of the difference image contour, obtaining the coordinates of the upper left corner and the width and height of the contour, and sequentially performing difference suppression include suppressing the size of the difference contour according to a preset image size threshold T, calculating the contour area, and comparing it with the preset image size threshold T, suppressing contours smaller than the specified size, which is expressed as: where F i Indicates the suppression flag, 1 means suppression is required; The edge of the difference contour is suppressed according to the preset boundary calculation threshold M, and the difference contour in the edge area is removed. The difference contour must meet the restriction condition to be considered to be in the valid area; Restrictions include: The left edge of the difference contour does not exceed the left edge limit, x l ≥M; The upper edge of the difference contour does not exceed the upper edge limit, y l ≥M; The right edge of the difference contour does not exceed the right edge limit, x l +w i ≤W i -M; The lower edge of the difference contour does not exceed the lower edge limit, y l +h i ≤H i -M; Among them, (x l ,y l ) is the coordinate of the upper left corner of the difference contour, (w i ,h i ) is the width and height of the difference contour, (W i , H i ) is the width and height of the specific area image I2 where there is a difference; The traversal of the difference image contour obtains the coordinates of the upper left corner of the contour and the width and height, and sequentially performs difference suppression, and further includes using an adaptive difference threshold algorithm to calculate a global threshold. Assume that the original image after denoising is B1, the difference image is B2, and the global mean and standard deviation are μ respectively. B1 and σ B1 : Among them, B1 i represents the gray value of the i-th pixel in the original image B1 after denoising, and N is the total number of pixels in the image; The global difference is calculated using the difference image B2: Among them, B2 i represents the gray value of the i-th pixel in the denoised difference image B2; Dynamically adjust the difference threshold according to the global features. Assuming the initial threshold is D0, the adjusted global threshold is D G It is expressed as: Where a is the global threshold adjustment coefficient; Use the adaptive difference threshold algorithm to calculate the local threshold, create a blank mask of the same size as the difference image Diff, and draw the current contour area on the mask; Use a mask to shield the non-difference contour area in the difference image Diff, set the pixel value of the non-difference area to 0, and only retain the pixel value of the difference contour area. Accumulate the non-zero pixel values ​​in the difference contour area and calculate the average to obtain the difference average value in the difference contour area; Compare with the difference threshold D. If it is not greater than D, it means that the difference does not meet expectations and needs to be suppressed. Traverse the suppressed difference contours, calculate the difference coincidence with the position information of the object to be filtered obtained by the initial screening of the difference image through the deep learning detection model, and determine whether the object to be filtered needs to be filtered according to the preset coincidence threshold C; The single difference coincidence calculation includes: Get the upper left corner coordinates (x1, y1) and lower right corner coordinates (x′1, y′1) of the difference contour R1, and the upper left corner coordinates (x2, y2) and lower right corner coordinates (x′2, y′2) of the object R2 to be filtered; By judging whether the coordinates of the upper left corner and the lower right corner of R1 are within the range of R2, it is determined whether R1 is completely contained in R2, which is expressed as: inside=x1≥x2∧y1≥y2∧x′1≤x′2∧y′1≤y′2 Calculate the overlapped area of ​​R1 and R2, and the coordinates of the upper left corner of the overlapped area: (x o ,y o )=(max(x1,x2),max(y1,y2)) The coordinates of the lower right corner of the overlapping area: (x′ o ,y′ o )=(min(x′1,x′2),min(y′1,y′2)) Calculate the area of ​​the overlapping area. If the width or height of the overlapping area is 0, it means there is no overlapping area. Otherwise, calculate the area of ​​the overlapping region, expressed as: area=(x′ o -x o )×(y′ o -and o ) Calculate the overlap, which is the ratio of the overlap area to the R1 area, expressed as: If they overlap and the overlap ratio is greater than the overlap threshold C, it means that the current difference contour is to be filtered and filtered. If a difference contour still exists after difference suppression, the contour information is drawn on the specific area image I2 where the difference exists to obtain the required specific area image difference.

2. The method for intelligently detecting and filtering differences in specific areas of images based on deep learning as claimed in claim 1, characterized in that: The initial screening of difference images by deep learning detection model includes collecting no-difference and difference image materials in specific areas, marking the materials, marking whether there are differences, using YOLOv5 to train the marked material set, and producing a specific area difference detection model. It is expressed as: in, is the boundary loss, λ bbox is the boundary loss weight, is the classification loss, λ cls is the classification loss weight, is the confidence loss, λ conf is the confidence loss weight, B p is the generated bounding box, B t is the input bounding box, is the input category label, is the probability that the input category label is the same as the model prediction label, t j is the true confidence of the input, is the model prediction confidence, N' represents the number of training samples used to calculate the classification loss, M' represents the number of training samples used to calculate the confidence loss, and C' is the number of labeled categories; The specific area difference detection model produced by training is used to identify the picture, and a preliminary screening is performed to determine whether there is a difference. After the judgment, a specific area image I1 without difference and a specific area image I2 with difference are obtained.

3. The method for intelligently detecting and filtering differences in specific areas of images based on deep learning as claimed in claim 2, characterized in that: The extraction of the objects to be filtered includes collecting image materials of the objects to be filtered according to the image filtering requirements, marking the objects to be filtered in the material images, training the marked material set, determining parameters and weights, inputting a specific area difference detection model for weight replacement to output a detection model for the objects to be filtered. The detection model for the objects to be filtered uses the model to identify I2 to determine whether there is an object to be filtered. If so, extracting the location information of the object to be filtered.

4. A system using the method for intelligently detecting and filtering differences in specific areas of images based on deep learning as described in any one of claims 1 to 3, characterized in that: It includes a preliminary screening module, an alignment module, and a difference determination module; The preliminary screening module is used to perform preliminary screening of the difference images and extract the objects to be filtered through the deep learning detection model; The alignment module is used to align the graphics without difference and the graphics with difference, calculate the difference of the grayscale images after registration and perform difference enhancement; The difference determination module is used to traverse the difference image contour, obtain the coordinates of the upper left corner and the width and height of the contour, and perform difference suppression in sequence.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for intelligently detecting and filtering the difference of specific areas of images based on deep learning as described in any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for intelligently detecting and filtering the difference of specific areas of a picture based on deep learning as described in any one of claims 1 to 3 are implemented.

Citation Information

Patent Citations

  • Traffic signal lamp recognition network training method and testing method based on visual ranging

    CN115546763A

  • Transmission conductor foreign matter detection method based on deep learning and frame difference method

    CN115761644A

  • Aircraft fuel tank redundancy automatic detection method based on YOLO

    CN117612081A