Infrared camera occlusion detection method, device, equipment and storage medium

Abstract

The present application relates to an infrared camera occlusion detection method, device, equipment and storage medium. A plurality of infrared sub-images are obtained by performing image scaling and image cropping processing on a current frame infrared image; median filtering processing is performed on each infrared sub-image to obtain a corresponding filtered infrared sub-image; edge detection processing is performed on each filtered infrared sub-image, and median filtering processing is performed on the obtained edge detection image to obtain a corresponding filtered edge detection image; edge point calculation is performed on each filtered edge detection image to obtain the corresponding current frame unit edge point size, and the infrared camera occlusion state is determined based on the current frame unit edge point size; compared with the prior art, the technical solution of the present application can improve the efficiency and accuracy of infrared camera occlusion detection.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to an infrared camera occlusion detection method, device, equipment and storage medium. Background Art

[0002] With the widespread application of camera technology in many fields, the real-time requirements for camera data are increasing to ensure that the camera can work reliably at any time and place. As a basic function to ensure the normal operation of the camera, camera occlusion detection can effectively reduce the time cost of manual monitoring and troubleshooting.

[0003] At present, the occlusion detection technology on the market mainly includes two methods: one is to determine the degree of occlusion by binarizing the image and calculating the area of ​​the foreground connected area; the other is to monitor the video stream on the network, restore the video, extract key frames for similarity comparison, and determine whether occlusion occurs; these camera occlusion detections are mainly performed on color cameras, and using their rich color information, occlusion detection can be achieved relatively easily.

[0004] However, for infrared cameras, due to the lack of color information, traditional color-based occlusion detection technology is difficult to achieve good results. In addition, complex algorithms consume a lot of computing resources, especially when processing infrared camera data, the demand for CPU resources is particularly significant; for example, on the rk1109 device, some mainstream occlusion area calculation and image similarity comparison methods consume up to 20% of CPU resources. Therefore, it is necessary to develop new technologies to solve the problems existing in infrared camera occlusion detection in order to achieve efficient and accurate occlusion detection. Summary of the invention

[0005] The present application provides an infrared camera occlusion detection method, device, equipment and storage medium, which can improve the efficiency and accuracy of infrared camera occlusion detection.

[0006] In a first aspect, the present application provides an infrared camera occlusion detection method, comprising: acquiring a current frame infrared image, and performing image scaling and image cropping processing on the current frame infrared image to obtain multiple infrared sub-images; performing median filtering processing on each infrared sub-image to obtain a corresponding filtered infrared sub-image; performing edge detection processing on each filtered infrared sub-image to obtain an edge detection image, and performing median filtering processing on the edge detection image to obtain a corresponding filtered edge detection image; performing edge point calculation on each filtered edge detection image to obtain the corresponding current frame unit edge point size, and determining the infrared camera occlusion state based on the current frame unit edge point size.

[0007] In a possible implementation, the current frame infrared image is subjected to image scaling and image cropping to obtain a plurality of infrared sub-images, specifically including: scaling the current frame infrared image based on a preset scaling ratio to obtain a scaled infrared image; copying the scaled infrared image to obtain a first scaled infrared image and a second scaled infrared image; cropping the first scaled infrared image to obtain a plurality of first cropped sub-images, and cropping the second scaled infrared image to obtain a plurality of second cropped sub-images; and integrating the plurality of first cropped sub-images and the plurality of second cropped sub-images to obtain a plurality of infrared sub-images.

[0008] In a possible implementation, the first scaled infrared image is cropped to obtain a plurality of first cropped sub-images, and the second scaled infrared image is cropped to obtain a plurality of second cropped sub-images, specifically including: obtaining the first corner point coordinates corresponding to the first scaled infrared image, and obtaining a preset horizontal cropping amount, and an image width value corresponding to the first scaled infrared image, determining the first cropping angle coordinates based on the first corner point coordinates, the horizontal cropping amount and the image width value, and performing horizontal cropping on the first scaled infrared image based on the first cropping angle coordinates to obtain a plurality of first cropped sub-images; obtaining the second corner point coordinates corresponding to the second scaled infrared image, and obtaining a preset vertical cropping amount, and an image length value corresponding to the second scaled infrared image, determining the second cropping angle coordinates based on the second corner point coordinates, the vertical cropping amount and the image length value, and performing vertical cropping on the second scaled infrared image based on the second cropping angle coordinates to obtain a plurality of second cropped sub-images.

[0009] In a possible implementation, the median filtering is performed on each infrared sub-image to obtain a corresponding filtered infrared sub-image, which specifically includes: traversing the pixel points in each infrared sub-image respectively, and during the traversal process, calculating the first neighborhood median corresponding to each pixel point based on a preset first smoothing operator, and updating the pixel value of the pixel point based on the first neighborhood median, until all the pixel points in the infrared sub-image are traversed to obtain a filtered infrared sub-image corresponding to each infrared sub-image.

[0010] In a possible implementation, the edge detection processing is performed on each filtered infrared sub-image to obtain an edge detection image, specifically including: setting algorithm parameters of a preset edge detection algorithm, wherein the algorithm parameters include a Sobel operator value, a high threshold number, a low threshold number, and a Euclidean norm; based on the Sobel operator value, respectively calculating the gradient and gradient direction corresponding to each pixel point in each filtered infrared sub-image, and based on the Euclidean norm, respectively calculating the gradient amplitude corresponding to the gradient of each pixel point; traversing each pixel point, and in the traversal process, determining whether the current pixel point is a local maximum based on the gradient amplitude and the gradient direction corresponding to the current pixel point value point, if yes, retain the current pixel point, otherwise, delete the current pixel point; obtain all retained pixel points, and compare the gradient amplitude corresponding to each retained pixel point with the high threshold number and the low threshold number, if the gradient amplitude is not less than the high threshold number, or the gradient amplitude is equal to the low threshold number, retain the target pixel point corresponding to the gradient amplitude, if the gradient amplitude is less than the low threshold number, or the gradient amplitude is greater than the low threshold number, and the gradient amplitude is less than the high threshold number, delete the target pixel point corresponding to the gradient amplitude; based on the retained target pixel points, generate an edge detection image corresponding to each filtered infrared sub-image.

[0011] In a possible implementation, the edge point calculation is performed on each filtered edge detection image to obtain the corresponding current frame unit edge point size, which specifically includes: counting the number of pixels corresponding to non-zero pixel values ​​in each filtered edge detection image to obtain the number of image edge points corresponding to each filtered edge detection image; calculating the total number of pixels corresponding to each filtered edge detection image based on the image width and image length corresponding to each filtered edge detection image; dividing the number of image edge points by the total number of pixels and the maximum value of the pixel range to obtain the current frame unit edge point size corresponding to each filtered edge detection image.

[0012] In a possible implementation, the infrared camera occlusion state is determined based on the current frame unit edge point size, specifically including: when it is detected that the current frame unit edge point size corresponding to any filtered edge detection image is smaller than a preset unit edge point size threshold, the previous frame unit edge point size corresponding to each filtered edge detection image is obtained respectively, and the current frame unit edge point size corresponding to each filtered edge detection image is compared with the previous frame unit edge point size; if the comparison result satisfies the infrared camera occlusion rule, the infrared camera occlusion state is determined to be that the infrared camera is occluded; otherwise, the infrared camera occlusion state is determined to be that the infrared camera is not occluded; when it is detected that the current frame unit edge point size corresponding to no filtered edge detection image is smaller than a preset unit edge point size threshold, the infrared camera occlusion state is determined to be that the infrared camera is not occluded.

[0013] In a possible implementation, an infrared camera occlusion detection device includes: an image processing module, a median filtering module, an edge detection module and an occlusion detection module; wherein the image processing module is used to collect a current frame infrared image, and perform image scaling and image cropping processing on the current frame infrared image to obtain multiple infrared sub-images; the median filtering module is used to perform median filtering processing on each infrared sub-image respectively to obtain a corresponding filtered infrared sub-image; the edge detection module is used to perform edge detection processing on each filtered infrared sub-image to obtain an edge detection image, and perform median filtering processing on the edge detection image to obtain a corresponding filtered edge detection image; the occlusion detection module is used to perform edge point calculation on each filtered edge detection image respectively to obtain the corresponding current frame unit edge point size, and determine the occlusion state of the infrared camera based on the current frame unit edge point size.

[0014] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program can implement the above method when executed by a processor.

[0016] The embodiments of the present application provide an infrared camera occlusion detection method, device, equipment and storage medium, which have the following advantages over the prior art:

[0017] By collecting the infrared image of the current frame, and performing image scaling and image cropping processing on the infrared image of the current frame, a plurality of infrared sub-images are obtained; a large-size image can be decomposed into a plurality of small-size sub-images, reducing the amount of data processed each time, thereby reducing the computational complexity and CPU resource consumption, and improving the subsequent detection efficiency; each infrared sub-image is subsequently subjected to median filtering processing to obtain a corresponding filtered infrared sub-image; each filtered infrared sub-image is subjected to edge detection processing to obtain an edge detection image, and the edge detection image is subjected to median filtering processing to obtain a corresponding filtered edge detection image; by performing median filtering twice on each cropped sub-image, the edge of useful information can be retained as much as possible, the interference of useless information on detection can be reduced, and the accuracy of detection can be improved; edge point calculation is performed on each filtered edge detection image to obtain the corresponding current frame unit edge point size, and the infrared camera occlusion state is determined based on the current frame unit edge point size; this operation can judge the image occlusion situation only by determining the current frame unit edge point size, and the process is simple to operate, and compared with the complex algorithm required in the prior art, it can reduce the occupation of computing resources and further improve the effectiveness of occlusion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0021] Figure 1 It is a flowchart of an embodiment of an infrared camera occlusion detection method provided by the present application;

[0022] Figure 2 It is a structural schematic diagram of an embodiment of an infrared camera occlusion detection device provided by the present application;

[0023] Figure 3 It is a structural schematic diagram of an electronic device provided by this application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0025] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0026] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0028] It should be further understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0029] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0030] Example 1, see Figure 1 , Figure 1FIG. 1 is a flow chart of an embodiment of an infrared camera occlusion detection method provided by the present application. Figure 1 As shown, the method includes steps 101 to 104, which are specifically as follows:

[0031] Step 101: Acquire a current frame of infrared image, and perform image scaling and image cropping processing on the current frame of infrared image to obtain a plurality of infrared sub-images.

[0032] In one embodiment, a current frame infrared image captured by an infrared camera is acquired; preferably, the current frame infrared image can be acquired from a video captured by the infrared camera.

[0033] In one embodiment, compared with ordinary cameras, the images captured by infrared cameras require special image processing technology to convert and display. These processing processes require additional computing resources, and infrared images usually require complex image processing operations such as super-resolution reconstruction and noise reduction. These operations involve a large number of mathematical operations and consume a large amount of CPU resources. Therefore, if infrared images are directly used for camera occlusion detection, the efficiency and effect of occlusion detection will be affected. Therefore, in this embodiment, while retaining the effective information of the image, the current frame infrared image is scaled and cropped, which can improve the detection efficiency of subsequent infrared camera occlusions.

[0034] In one embodiment, the current frame infrared image is scaled based on a preset scaling ratio to obtain a scaled infrared image; the scaled infrared image is copied to obtain a first scaled infrared image and a second scaled infrared image; the first scaled infrared image is cropped to obtain a plurality of first cropped sub-images, and the second scaled infrared image is cropped to obtain a plurality of second cropped sub-images; the plurality of first cropped sub-images and the plurality of second cropped sub-images are integrated to obtain a plurality of infrared sub-images.

[0035] Specifically, the image size of the current frame infrared image is obtained, and the image ratio of the image size to the preset image size is calculated, and the image ratio is set as the preset scaling ratio, and the current frame infrared image is scaled.

[0036] Preferably, if the image size of the current frame infrared image is 640*640 and the preset image size is 320*320, the preset scaling ratio is 2:1; and based on the preset scaling ratio, the current frame infrared image of 640*640 is scaled to a scaled infrared image of 320*320, which can save CPU resource consumption for subsequent steps by reducing the image size.

[0037] Preferably, the cv::resize function is also called, wherein the first parameter of the cv::resize function is the input image, the second parameter is the output image, and the third parameter is the size of the specified scaled image. Based on the cv::resize function, the current frame infrared image is directly scaled to obtain a scaled infrared image.

[0038] Specifically, after scaling the infrared image of the current frame, the scaled infrared image is also copied and cropped to decompose the large-size scaled infrared image into multiple small-size infrared sub-images. Each infrared sub-image can be processed independently, which can reduce the amount of data processed each time, reduce CPU resource consumption, and improve processing speed.

[0039] Specifically, the first zoomed infrared image and the second zoomed infrared image are the same image.

[0040] Specifically, since camera occlusion may occur in any part of the image and different types of occlusion may exist in different areas, by copying the same scaled infrared image and decomposing the copied image using different cropping methods, it can be ensured that the detection algorithm can adapt to various occlusion situations.

[0041] In one embodiment, the coordinates of the first corner point corresponding to the first scaled infrared image are obtained, and the preset horizontal cropping number and the image width value corresponding to the first scaled infrared image are obtained; based on the first corner point coordinates, the horizontal cropping number and the image width value, the first cropping angle coordinates are determined; and based on the first cropping angle coordinates, the first scaled infrared image is horizontally cropped to obtain multiple first cropped sub-images.

[0042] Preferably, the plurality of first cropped sub-images include an upper cropped sub-image and a lower cropped sub-image.

[0043] Specifically, the first corner point coordinates are the upper left corner coordinates of the first zoomed infrared image, and the first cropping corner coordinates are the first fixed point for horizontal cropping.

[0044] Specifically, based on the number of horizontal cropping, the image width values ​​corresponding to the first scaled infrared image are evenly divided to obtain the image width mean value corresponding to the horizontally cropped sub-image, and the first corner point coordinates are set as the zero point coordinates. Based on the image width mean value, the y-axes of the first corner point coordinates are superimposed in sequence, and the first coordinate value obtained after each superposition is obtained until the y-axes of the first corner point coordinates obtained after superposition are not less than the image width value corresponding to the first scaled infrared image, and the superposition process is stopped; the last first coordinate is deleted, and all retained first coordinate values ​​are used as the first cropping corner coordinates.

[0045] As an example, if the first zoomed infrared image needs to be horizontally cropped into two parts, the corresponding horizontal cropping number is 1. Since the image width value corresponding to the first zoomed infrared image is 320, at this time, the image width average value corresponding to the horizontally cropped sub-image is 160, and the first corner point coordinate is set to (0, 0). Based on the image width average value 160, the y-axis of the first corner point coordinate is superimposed to obtain (0, 160) and (0, 320). Since 320 is the same as the image width value corresponding to the first zoomed infrared image, the superposition is stopped, and the last first coordinate (0, 320) is deleted to obtain the first cropping angle coordinate (0, 160). Based on the first cropping angle coordinate (0, 160), the first zoomed infrared image is horizontally cropped to obtain multiple first cropped sub-images, which are the upper zoomed infrared image and the lower zoomed infrared image.

[0046] Specifically, the horizontal cropping process refers to cropping along the x-axis direction of the first zoomed infrared image.

[0047] Preferably, the first cropping corner coordinates may also be the upper left corner coordinates and the lower right corner coordinates corresponding to the first cropped sub-image.

[0048] In one embodiment, the second corner point coordinates corresponding to the second scaled infrared image are obtained, and the preset vertical cropping number and the image length value corresponding to the second scaled infrared image are obtained; based on the second corner point coordinates, the vertical cropping number and the image length value, the second cropping angle coordinates are determined; and based on the second cropping angle coordinates, the second scaled infrared image is vertically cropped to obtain multiple second cropped sub-images.

[0049] Specifically, the second corner point coordinates are the coordinates of the upper left corner of the second zoomed infrared image, and the second cropping corner coordinates are the second fixed point for vertically cropping the second zoomed infrared image.

[0050] Specifically, based on the number of vertical cropping, the image length values ​​corresponding to the second scaled infrared image are evenly divided to obtain the mean image length corresponding to the vertically cropped sub-image, and the second corner point coordinates are set as the zero point coordinates. Based on the mean image length, the x-axis of the second corner point coordinates are superimposed in sequence, and the second coordinate value obtained after each superposition is obtained until the x-axis of the second corner point coordinates obtained after superposition is not less than the image length value corresponding to the second scaled infrared image, and the superposition process is stopped; the last second coordinate is deleted, and all retained second coordinate values ​​are used as the second cropping corner coordinates.

[0051] As an example, if the second zoomed infrared image needs to be vertically cropped into two parts, the corresponding vertical cropping number is 1. Since the image length value corresponding to the second zoomed infrared image is 320, at this time, the image length average value corresponding to the vertically cropped sub-image is 160, and the second corner point coordinate is set to (0, 0). Based on the image length average value 160, the x-axis of the second corner point coordinate is superimposed to obtain (160, 0) and (320, 0). Since 320 is the same as the image length value corresponding to the second zoomed infrared image, the superposition is stopped, and the second coordinate (320, 0) at the end is deleted to obtain the second cropping angle coordinate (160, 0). Based on the second cropping angle coordinate (160, 0), the second zoomed infrared image is vertically cropped to obtain multiple second cropped sub-images, which are the left zoomed infrared image and the right zoomed infrared image.

[0052] Specifically, the vertical cropping process refers to cropping along the y-axis direction of the second zoomed infrared image.

[0053] Step 102: Perform median filtering on each infrared sub-image to obtain a corresponding filtered infrared sub-image.

[0054] In one embodiment, the pixel points in each infrared sub-image are traversed separately, and during the traversal process, the first neighborhood median corresponding to each pixel point is calculated based on a preset first smoothing operator, and the pixel value of the pixel point is updated based on the first neighborhood median until all the pixel points in the infrared sub-image are traversed to obtain a filtered infrared sub-image corresponding to each infrared sub-image.

[0055] Specifically, median filtering is performed on each infrared sub-image through the median algorithm cv::medianBlur(images1, images2, 3); images1 is the original image, that is, the infrared sub-image, images2 is the image after median filtering, that is, the filtered infrared sub-image, and parameter 3 refers to setting the size of the smoothing operator to 3. Based on the median algorithm, low noise filtering is achieved for the infrared sub-image, which retains the original information of the image while playing a certain role in removing interference information.

[0056] Specifically, for each pixel in the infrared sub-image, determine its neighborhood, that is, the 3x3 area centered on the pixel, extract the pixel values ​​corresponding to all pixels in the neighborhood, sort the pixel values ​​corresponding to all pixels by size, get the middle pixel value, and use the middle pixel value as the first neighborhood median, and based on the first neighborhood median, update the pixel value of the current pixel, repeat the above process until all pixels in the infrared sub-image are traversed and updated to obtain a filtered infrared sub-image.

[0057] Step 103: performing edge detection processing on each filtered infrared sub-image to obtain an edge detection image, and performing median filtering processing on the edge detection image to obtain a corresponding filtered edge detection image.

[0058] In one embodiment, algorithm parameters of a preset edge detection algorithm are set, wherein the algorithm parameters include a Sobel operator value, a high threshold number, a low threshold number, and a Euclidean norm.

[0059] Specifically, the Sobel operator value is set to 3 to calculate the gradient of each pixel in the image.

[0060] Specifically, the high threshold value is set to 90 for determining a strong edge, and the low threshold value is set to 35 for determining a weak edge.

[0061] Specifically, the Euclidean norm L2 is used to calculate the magnitude of the gradient, which is the square root of the sum of the squares of the gradient vector in the x and y directions.

[0062] In one embodiment, based on the Sobel operator value, the gradient and gradient direction corresponding to each pixel in each filtered infrared sub-image are calculated respectively, and based on the Euclidean norm, the gradient amplitude corresponding to the gradient of each pixel is calculated respectively; each pixel is traversed, and during the traversal process, based on the gradient amplitude and the gradient direction corresponding to the current pixel, it is determined whether the current pixel is a local maximum point, if so, the current pixel is retained, otherwise, the current pixel is deleted; all retained pixels are obtained, and the gradient amplitude corresponding to each retained pixel is compared with the high threshold number and the low threshold number, if the gradient amplitude is not less than the high threshold number, or the gradient amplitude is equal to the low threshold number, then the target pixel corresponding to the gradient amplitude is retained, if the gradient amplitude is less than the low threshold number, or the gradient amplitude is greater than the low threshold number, and the gradient amplitude is less than the high threshold number, then the target pixel corresponding to the gradient amplitude is deleted; based on the retained target pixels, an edge detection image corresponding to each filtered infrared sub-image is generated.

[0063] Specifically, after obtaining the gradient amplitude corresponding to each pixel in each filtered infrared sub-image, the pixels are traversed one by one to determine whether the gradient amplitude of the current pixel is the maximum value among the surrounding pixels with the same gradient direction, and whether to suppress the point is determined based on the judgment result. If the point is the local maximum in the positive / negative gradient direction, the pixel is retained. If not, the pixel is suppressed (returned to zero); after the above processing, for several edge points in the same direction, basically only one is retained, thus achieving the purpose of edge refinement.

[0064] Specifically, after completing the above suppression, since the strong edge of the image is already in the currently acquired edge image, however, some virtual edges may also be in the edge image. These virtual edges may be generated by the real image or by noise. Therefore, the virtual edges generated by noise must be removed, and the attributes of the edge are judged based on the relationship between the gradient amplitude of the current edge pixel and the high threshold number and the low threshold number.

[0065] In one embodiment, when the edge detection image is subjected to median filtering to obtain the corresponding filtered edge detection image, the pixel points in each edge detection image are traversed respectively, and during the traversal process, the second neighborhood median corresponding to each pixel point is calculated based on a preset second smoothing operator, and the pixel value of the pixel point is updated based on the second neighborhood median until all the pixel points in the edge detection image are traversed to obtain the filtered edge detection image corresponding to each edge detection image.

[0066] Specifically, median filtering is performed on each edge detection image through the median algorithm cv::medianBlur(edges,edges_me,5); where edges is the original image, i.e., the edge detection image, edges_me is the image after median filtering, i.e., the filtered edge detection image, and parameter 5 refers to setting the size of the smoothing operator to 5, so as to implement large noise filtering on the edge detection image based on the median algorithm.

[0067] Specifically, for each pixel in the edge detection image, determine its neighborhood, that is, a 5x5 area centered on the pixel, extract pixel values ​​corresponding to all pixels in the neighborhood, sort the pixel values ​​corresponding to all pixels by size, obtain an intermediate pixel value, and use the intermediate pixel value as the median of the second neighborhood, and based on the median of the second neighborhood, update the pixel value of the current pixel, and repeat the above process until all pixels in the edge detection image are traversed and updated to obtain a filtered edge detection image.

[0068] Step 104: Calculate edge points of each filtered edge detection image respectively to obtain the corresponding current frame unit edge point size, and determine the infrared camera occlusion state based on the current frame unit edge point size.

[0069] In one embodiment, the number of image edge points corresponding to each filtered edge detection image is obtained by respectively counting the number of pixels corresponding to non-zero pixel values ​​in each filtered edge detection image; based on the image width and image length corresponding to each filtered edge detection image, the total number of pixels corresponding to each filtered edge detection image is calculated; the number of image edge points is divided by the total number of pixels and the maximum value of the pixel range to obtain the current frame unit edge point size corresponding to each filtered edge detection image.

[0070] Specifically, for each filtered edge detection image, the cv::sum function is used to calculate the sum of all pixels with non-zero pixel values ​​in the image to obtain the number of pixels corresponding to the non-zero pixel values, that is, the number of edge points in the image.

[0071] Specifically, the total number of pixels of each filtered edge detection image can be calculated by multiplying its image width by its image length.

[0072] Specifically, when the number of image edge points is divided by the total number of pixels and the maximum value of the pixel range, the number of image edge points is divided by the product of the total number of pixels and the maximum value of the pixel range to obtain the current frame unit edge point size corresponding to each filtered edge detection image.

[0073] In one embodiment, when determining the occlusion state of the infrared camera based on the unit edge point size of the current frame, when it is detected that the unit edge point size of the current frame corresponding to any filtered edge detection image is smaller than a preset unit edge point size threshold, the unit edge point size of the previous frame corresponding to each filtered edge detection image is obtained respectively, and the unit edge point size of the current frame corresponding to each filtered edge detection image and the unit edge point size of the previous frame are compared and processed. If the comparison result meets the infrared camera occlusion rule, the occlusion state of the infrared camera is determined to be that the infrared camera is occluded, otherwise, the occlusion state of the infrared camera is determined to be that the infrared camera is not occluded; when it is detected that the unit edge point size of the current frame corresponding to no filtered edge detection image is smaller than the preset unit edge point size threshold, the occlusion state of the infrared camera is determined to be that the infrared camera is not occluded.

[0074] Preferably, the preset unit edge point size threshold is 150.

[0075] In one embodiment, when comparing the unit edge point size of the current frame and the unit edge point size of the previous frame corresponding to each filtered edge detection image, a first difference is obtained by performing a difference calculation on the unit edge point size of the current frame and the unit edge point size of the previous frame corresponding to each filtered edge detection image. If the first difference is not greater than a preset target difference, it is determined that the comparison result satisfies the infrared camera occlusion rule; otherwise, it is determined that the comparison result does not satisfy the infrared camera occlusion rule.

[0076] Preferably, the preset target difference is 85.

[0077] Preferably, in the case of an infrared camera, if the first difference is greater than the preset target difference, it is considered that the difference is caused by the movement of the level object due to the imaging of the infrared camera, and not by occlusion. Therefore, by setting the target difference to 85, the occlusion detection error of the infrared camera can be further avoided and the accuracy of occlusion detection can be improved.

[0078] See also Figure 2 , Figure 2 1 is a schematic diagram of the structure of an embodiment of an infrared camera occlusion detection device provided by the present application. Corresponding to the above infrared camera occlusion detection method, the present application also provides an infrared camera occlusion detection device. The infrared camera occlusion detection device includes a module for executing the above infrared camera occlusion detection method, and the infrared camera occlusion detection device can be configured in a desktop computer, a tablet computer, a laptop computer, and other terminals. Specifically, the infrared camera occlusion detection device includes an image processing module 201, a median filtering module 202, an edge detection module 203, and an occlusion detection module 204.

[0079] The image processing module 201 is used to collect a current frame of infrared image, and perform image scaling and image cropping processing on the current frame of infrared image to obtain a plurality of infrared sub-images.

[0080] The median filtering module 202 is used to perform median filtering on each infrared sub-image to obtain a corresponding filtered infrared sub-image.

[0081] The edge detection module 203 is used to perform edge detection processing on each filtered infrared sub-image to obtain an edge detection image, and perform median filtering processing on the edge detection image to obtain a corresponding filtered edge detection image.

[0082] The occlusion detection module 204 is used to calculate edge points of each filtered edge detection image respectively to obtain the corresponding current frame unit edge point size, and determine the occlusion state of the infrared camera based on the current frame unit edge point size.

[0083] In one embodiment, the image processing module 201 is used to perform image scaling and image cropping on the current frame infrared image to obtain multiple infrared sub-images, specifically including: scaling the current frame infrared image based on a preset scaling ratio to obtain a scaled infrared image; copying the scaled infrared image to obtain a first scaled infrared image and a second scaled infrared image; cropping the first scaled infrared image to obtain multiple first cropped sub-images, and cropping the second scaled infrared image to obtain multiple second cropped sub-images; integrating the multiple first cropped sub-images and the multiple second cropped sub-images to obtain multiple infrared sub-images.

[0084] In one embodiment, the image processing module 201 is used to crop the first scaled infrared image to obtain multiple first cropped sub-images, and to crop the second scaled infrared image to obtain multiple second cropped sub-images, specifically including: obtaining the first corner point coordinates corresponding to the first scaled infrared image, and obtaining a preset horizontal cropping number, and an image width value corresponding to the first scaled infrared image, determining the first cropping angle coordinates based on the first corner point coordinates, the horizontal cropping number and the image width value, and performing horizontal cropping on the first scaled infrared image based on the first cropping angle coordinates to obtain multiple first cropped sub-images; obtaining the second corner point coordinates corresponding to the second scaled infrared image, and obtaining a preset vertical cropping number, and an image length value corresponding to the second scaled infrared image, determining the second cropping angle coordinates based on the second corner point coordinates, the vertical cropping number and the image length value, and performing vertical cropping on the second scaled infrared image based on the second cropping angle coordinates to obtain multiple second cropped sub-images.

[0085] In one embodiment, the median filtering module 202 is used to perform median filtering on each infrared sub-image respectively to obtain a corresponding filtered infrared sub-image, specifically including: traversing the pixel points in each infrared sub-image respectively, and during the traversal process, calculating the first neighborhood median corresponding to each pixel point based on a preset first smoothing operator, and updating the pixel value of the pixel point based on the first neighborhood median, until all the pixel points in the infrared sub-image are traversed to obtain a filtered infrared sub-image corresponding to each infrared sub-image.

[0086] In one embodiment, the edge detection module 203 is used to perform edge detection processing on each filtered infrared sub-image to obtain an edge detection image, specifically including: setting algorithm parameters of a preset edge detection algorithm, wherein the algorithm parameters include a Sobel operator value, a high threshold number, a low threshold number and a Euclidean norm; based on the Sobel operator value, respectively calculating the gradient corresponding to each pixel point in each filtered infrared sub-image, and based on the Euclidean norm, respectively calculating the gradient amplitude corresponding to the gradient of each pixel point; comparing the gradient amplitude with the high threshold number and the low threshold number, if the gradient amplitude is not less than the high threshold number, or the gradient amplitude is equal to the low threshold number, then retaining the target pixel point corresponding to the gradient amplitude; if the gradient amplitude is less than the low threshold number, or the gradient amplitude is greater than the low threshold number, and the gradient amplitude is less than the high threshold number, then deleting the target pixel point corresponding to the gradient amplitude; based on the retained target pixel points, generating an edge detection image corresponding to each filtered infrared sub-image.

[0087] In one embodiment, the occlusion detection module 204 is used to perform edge point calculations on each filtered edge detection image respectively to obtain the corresponding current frame unit edge point size, specifically including: counting the number of pixels corresponding to non-zero pixel values ​​in each filtered edge detection image respectively to obtain the number of image edge points corresponding to each filtered edge detection image; calculating the total number of pixels corresponding to each filtered edge detection image based on the image width and image length corresponding to each filtered edge detection image; dividing the number of image edge points by the total number of pixels and the maximum value of the pixel range to obtain the current frame unit edge point size corresponding to each filtered edge detection image.

[0088] In one embodiment, the occlusion detection module 204 is used to determine the occlusion state of the infrared camera based on the unit edge point size of the current frame, specifically including: when it is detected that the unit edge point size of the current frame corresponding to any filtered edge detection image is smaller than a preset unit edge point size threshold, respectively obtain the unit edge point size of the previous frame corresponding to each filtered edge detection image, and compare the unit edge point size of the current frame corresponding to each filtered edge detection image with the unit edge point size of the previous frame; if the comparison result satisfies the infrared camera occlusion rule, then the occlusion state of the infrared camera is determined to be that the infrared camera is occluded; otherwise, the occlusion state of the infrared camera is determined to be that the infrared camera is not occluded; when it is detected that the unit edge point size of the current frame corresponding to no filtered edge detection image is smaller than the preset unit edge point size threshold, then the occlusion state of the infrared camera is determined to be that the infrared camera is not occluded.

[0089] The infrared camera occlusion detection device can implement the infrared camera occlusion detection method of the method embodiment. The options in the method embodiment are also applicable to this embodiment and will not be described in detail here.

[0090] like Figure 3 As shown, Figure 3 It is a structural diagram of an electronic device provided by the present application; it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114, and the memory 113 is used to store computer programs.

[0091] In one embodiment of the present application, the processor 111 is used to implement the infrared camera occlusion detection method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 113.

[0092] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0093] Therefore, an embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the infrared camera occlusion detection method provided in any of the aforementioned method embodiments are implemented.

[0094] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which can store program codes. The computer-readable storage medium can be non-volatile or volatile.

[0095] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0096] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0097] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0099] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0101] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for detecting occlusion of an infrared camera, characterized in that: include: Acquire a current frame infrared image, and scale the current frame infrared image based on a preset scaling ratio to obtain a scaled infrared image; Copying the scaled infrared image to obtain a first scaled infrared image and a second scaled infrared image; cropping the first scaled infrared image to obtain a plurality of first cropped sub-images, and cropping the second scaled infrared image to obtain a plurality of second cropped sub-images; Integrate the plurality of first cropped sub-images and the plurality of second cropped sub-images to obtain a plurality of infrared sub-images, wherein the plurality of first cropped sub-images are upper zoomed infrared images and lower zoomed infrared images, and the plurality of second cropped sub-images are left zoomed infrared images and right zoomed infrared images; Perform median filtering on each infrared sub-image respectively to obtain a corresponding filtered infrared sub-image; Performing edge detection processing on each filtered infrared sub-image to obtain an edge detection image, and performing median filtering processing on the edge detection image to obtain a corresponding filtered edge detection image; Edge point calculation is performed on each filtered edge detection image respectively to obtain the corresponding current frame unit edge point size, and the infrared camera occlusion state is determined based on the current frame unit edge point size.

2. The method according to claim 1, characterized in that The first scaled infrared image is cropped to obtain a plurality of first cropped sub-images, and the second scaled infrared image is cropped to obtain a plurality of second cropped sub-images, specifically comprising: Acquire the coordinates of a first corner point corresponding to the first zoomed infrared image, and acquire a preset horizontal cropping amount, and an image width value corresponding to the first zoomed infrared image, determine a first cropping angle coordinate based on the first corner point coordinates, the horizontal cropping amount, and the image width value, and perform horizontal cropping processing on the first zoomed infrared image based on the first cropping angle coordinates to obtain a plurality of first cropped sub-images; Obtain the second corner point coordinates corresponding to the second scaled infrared image, and obtain the preset vertical cropping number and the image length value corresponding to the second scaled infrared image; determine the second cropping angle coordinates based on the second corner point coordinates, the vertical cropping number and the image length value; and perform vertical cropping processing on the second scaled infrared image based on the second cropping angle coordinates to obtain multiple second cropped sub-images.

3. The method according to claim 1, characterized in that: The median filtering is performed on each infrared sub-image to obtain a corresponding filtered infrared sub-image, specifically including: The pixel points in each infrared sub-image are traversed respectively, and during the traversal process, the first neighborhood median corresponding to each pixel point is calculated based on the preset first smoothing operator, and the pixel value of the pixel point is updated based on the first neighborhood median, until all the pixel points in the infrared sub-image are traversed, and a filtered infrared sub-image corresponding to each infrared sub-image is obtained.

4. The method according to claim 1, characterized in that: The edge detection process is performed on each filtered infrared sub-image to obtain an edge detection image, specifically comprising: Setting algorithm parameters of a preset edge detection algorithm, wherein the algorithm parameters include a Sobel operator value, a high threshold number, a low threshold number, and a Euclidean norm; Based on the Sobel operator value, respectively calculating the gradient and gradient direction corresponding to each pixel point in each filtered infrared sub-image, and based on the Euclidean norm, respectively calculating the gradient amplitude corresponding to the gradient of each pixel point; Traversing each pixel point, and during the traversal process, determining whether the current pixel point is a local maximum point based on the gradient magnitude and the gradient direction corresponding to the current pixel point, if so, retaining the current pixel point, otherwise, deleting the current pixel point; Acquire all reserved pixel points, and compare the gradient amplitude corresponding to each reserved pixel point with the high threshold number and the low threshold number; if the gradient amplitude is not less than the high threshold number, or the gradient amplitude is equal to the low threshold number, retain the target pixel point corresponding to the gradient amplitude; if the gradient amplitude is less than the low threshold number, or the gradient amplitude is greater than the low threshold number and the gradient amplitude is less than the high threshold number, delete the target pixel point corresponding to the gradient amplitude; Based on the retained target pixel points, an edge detection image corresponding to each filtered infrared sub-image is generated.

5. The method according to claim 1, characterized in that: The edge point calculation is performed on each filtered edge detection image to obtain the corresponding current frame unit edge point size, specifically including: Count the number of pixel points corresponding to non-zero pixel values ​​in each filtered edge detection image respectively, and obtain the number of image edge points corresponding to each filtered edge detection image; Calculate the total number of pixels corresponding to each filtered edge detection image based on the image width and image length corresponding to each filtered edge detection image; The number of edge points of the image is divided by the total number of pixel points and the maximum value of the pixel range to obtain the current frame unit edge point size corresponding to each filtered edge detection image.

6. The method according to claim 1, characterized in that: The determining the infrared camera occlusion state based on the unit edge point size of the current frame specifically includes: When it is detected that the unit edge point size of the current frame corresponding to any filtered edge detection image is smaller than the preset unit edge point size threshold, the unit edge point size of the previous frame corresponding to each filtered edge detection image is obtained respectively, and the unit edge point size of the current frame corresponding to each filtered edge detection image is compared with the unit edge point size of the previous frame. If the comparison result satisfies the infrared camera occlusion rule, the infrared camera occlusion state is determined to be that the infrared camera is occluded, otherwise, the infrared camera occlusion state is determined to be that the infrared camera is not occluded; When it is detected that there is no unit edge point size of the current frame corresponding to any filtered edge detection image that is smaller than a preset unit edge point size threshold, it is determined that the infrared camera occlusion state is that the infrared camera is not occluded.

7. An infrared camera occlusion detection device, characterized in that: include: Image processing module, median filtering module, edge detection module and occlusion detection module; Wherein, the image processing module is used to collect the infrared image of the current frame, and scale the infrared image of the current frame based on a preset scaling ratio to obtain a scaled infrared image; copy the scaled infrared image to obtain a first scaled infrared image and a second scaled infrared image; crop the first scaled infrared image to obtain a plurality of first cropped sub-images, and crop the second scaled infrared image to obtain a plurality of second cropped sub-images; integrate the plurality of first cropped sub-images and the plurality of second cropped sub-images to obtain a plurality of infrared sub-images, wherein the plurality of first cropped sub-images are an upper scaled infrared image and a lower scaled infrared image, and the plurality of second cropped sub-images are a left scaled infrared image and a right scaled infrared image; The median filtering module is used to perform median filtering on each infrared sub-image to obtain a corresponding filtered infrared sub-image; The edge detection module is used to perform edge detection processing on each filtered infrared sub-image to obtain an edge detection image, and perform median filtering processing on the edge detection image to obtain a corresponding filtered edge detection image; The occlusion detection module is used to calculate edge points of each filtered edge detection image respectively, obtain the corresponding current frame unit edge point size, and determine the occlusion state of the infrared camera based on the current frame unit edge point size.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 can be implemented.

Images