Method and system for target detection in night scene, and storage medium

Abstract

Disclosed in the present invention are a method and system for target detection in a night scene, and a storage medium, which are applied to the technical field of computer vision, and are capable of effectively improving the accuracy of living-body target detection in night scenes, and improving the accuracy of positioning identified targets. The method comprises: performing preset decoding on a video to be subjected to detection, so as to obtain video frame images; performing grayscale conversion on the video frame images, and storing same in a grayscale frame array; when it is determined that the length of the grayscale frame array is greater than a first length threshold, acquiring grayscale frame image pairs from the grayscale frame array to perform differentiation operations, so as to obtain residual frame images; when pixel values of the residual frame images meet a preset pixel condition, storing the residual frame images in a second image array, so as to obtain a residual frame array; when it is determined that the length of the residual frame array is greater than a second length threshold, performing analysis on the basis of pixel points of the residual frame images in the residual frame array, so as to obtain target pixel coordinate points; and performing preset clustering processing on the basis of the target pixel coordinate points, so as to obtain a target identification result.

Description

Night scene target detection method, system and storage medium Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, system and storage medium for detecting targets in nighttime scenes. Background Art

[0002] With the development and advancement of video analysis and artificial intelligence technologies, more and more events can be detected and identified in real time through video surveillance. For example, in the kitchen monitoring of catering establishments, video analysis can detect rats in the surveillance scene, record their tracks, detect them in advance, and issue an alarm. Rats entering the detection area can be identified. If a rat is detected, an alarm can be issued and the alarm signal can be simultaneously pushed to management personnel, greatly improving the hygiene and safety level of the kitchen. However, in related technologies, in nighttime scenes, due to the accuracy of the detection algorithm, objects in the picture are often misidentified as the target, resulting in false alarms. In addition, the accuracy of deep learning target detection methods relies on large datasets. However, datasets for detecting live targets at night are often difficult to collect, such as datasets of rat activity in nighttime scenes. Searching for them in relevant surveillance videos is time-consuming and labor-intensive. Therefore, the above technical issues need to be urgently addressed.

[0003] Summary of the Invention

[0004] In order to solve at least one of the above technical problems, the present invention proposes a night scene target detection method, system and storage medium, which can effectively improve the accuracy of living target detection in night scenes and improve the positioning accuracy of identified targets.

[0005] In one aspect, an embodiment of the present invention provides a method for detecting a target in a nighttime scene, comprising the following steps:

[0006] Perform preset decoding on the video to be detected to obtain a video frame image;

[0007] Convert the video frame image into grayscale and store it in a first image array to obtain a grayscale frame array;

[0008] When it is determined that the array length of the grayscale frame array is greater than a first threshold length, obtaining a grayscale frame image pair from the grayscale frame array; wherein the grayscale frame image pair includes a first frame image and a second frame image, and the first frame image is a previous frame image of the second frame image;

[0009] Performing a difference operation on the first frame image and the second frame image to obtain a residual frame image;

[0010] When the pixel value of the residual frame image meets the preset pixel condition, the residual frame image is stored in the second image array to obtain a residual frame array;

[0011] When it is determined that the array length of the residual frame array is greater than a second threshold length, performing calculation and analysis based on the pixel points of each residual frame image in the residual frame array to obtain the target pixel coordinate point;

[0012] A preset clustering process is performed according to the target pixel coordinate points to obtain a target recognition result.

[0013] According to some embodiments of the present invention, when the pixel value of the residual frame image meets a preset pixel condition, storing the residual frame image in a second image array to obtain a residual frame array includes:

[0014] Binarizing the residual frame image to obtain a binary frame image;

[0015] Performing pixel mean calculation on the binary frame image to obtain pixel mean data;

[0016] When it is determined that the pixel mean data is less than a preset pixel threshold, the residual frame image is stored in the second image array to obtain the residual frame array.

[0017] According to some embodiments of the present invention, performing calculation and analysis on pixel points of each residual frame image in the residual frame array to obtain target pixel coordinate points includes:

[0018] Acquire a preset number of the residual frame images from the residual frame array to obtain a residual image group;

[0019] Performing an AND operation on the corresponding pixel points of each residual frame image in the residual image group to obtain a pixel matrix;

[0020] The target pixel coordinate point is determined according to the pixel matrix.

[0021] According to some embodiments of the present invention, determining the target pixel coordinate point according to the pixel matrix includes:

[0022] When it is determined that the element values ​​of the pixel matrix are not all zero, the pixel coordinates corresponding to the element values ​​that are not zero in the pixel matrix are obtained to obtain the target pixel coordinate point.

[0023] According to some embodiments of the present invention, performing a preset clustering process based on the target pixel coordinate points to obtain a target recognition result includes:

[0024] According to the target pixel coordinate points, a preset search algorithm is used to find the contour points of the non-zero connected area to obtain several groups of target contour points;

[0025] Calculating the coordinate mean of each group of target contour points to obtain the corresponding approximate center point;

[0026] Calculating the Euclidean distance between the approximate center points to obtain center distance data;

[0027] When it is determined that the center distance data is less than a preset distance threshold, establishing an adjacency table;

[0028] The target recognition result is obtained by analyzing the adjacency table using a preset priority algorithm.

[0029] According to some embodiments of the present invention, analyzing the adjacency table using a preset priority algorithm to obtain the target recognition result includes:

[0030] Traversing the adjacency list using a depth-first algorithm, grouping the approximate center points to obtain center point grouping data;

[0031] Grouping the target contour points according to the center point grouping data and calculating corresponding corner points to obtain corner point data;

[0032] The circumscribed rectangle is calculated based on the corner point data to obtain the target recognition result.

[0033] According to some embodiments of the present invention, the method further comprises:

[0034] The pixel value of the preset display area of ​​each grayscale image in the grayscale frame array is set to 0; wherein the preset display area includes a time display area.

[0035] On the other hand, an embodiment of the present invention further provides a night scene target detection system, comprising:

[0036] The first module is used to perform preset decoding on the video to be detected to obtain a video frame image;

[0037] The second module is configured to perform grayscale conversion on the video frame image and store the grayscale image in the first image array to obtain a grayscale frame array;

[0038] A third module is configured to obtain a grayscale frame image pair from the grayscale frame array when it is determined that the array length of the grayscale frame array is greater than a first threshold length; wherein the grayscale frame image pair includes a first frame image and a second frame image, and the first frame image is a previous frame image of the second frame image;

[0039] a fourth module, configured to perform a difference operation on the first frame image and the second frame image to obtain a residual frame image;

[0040] A fifth module is configured to store the residual frame image into a second image array to obtain a residual frame array when the pixel value of the residual frame image meets a preset pixel condition;

[0041] A sixth module is configured to, when it is determined that the array length of the residual frame array is greater than a second threshold length, perform calculation and analysis based on the pixel points of each residual frame image in the residual frame array to obtain a target pixel coordinate point;

[0042] The seventh module is used to perform preset clustering processing according to the target pixel coordinate points to obtain a target recognition result.

[0043] On the other hand, an embodiment of the present invention further provides a night scene target detection system, comprising:

[0044] at least one processor;

[0045] at least one memory for storing at least one program;

[0046] When the at least one program is executed by the at least one processor, the at least one processor implements the night scene target detection method as described in the above embodiment.

[0047] On the other hand, an embodiment of the present invention further provides a computer storage medium storing a program executable by a processor. When the program is executed by the processor, it is used to implement the night scene target detection method as described in the above embodiment.

[0048] According to an embodiment of the present invention, a method, system, and storage medium for nighttime scene target detection have at least the following beneficial effects: The embodiment of the present invention first performs a preset decoding on a video to be detected to obtain a video frame image, performs grayscale conversion on the video frame image, and stores the grayscale frame image in a first image array to construct a grayscale frame array. Then, when the array length of the grayscale frame array is greater than a first threshold length, that is, when the number of grayscale frame images stored in the grayscale frame array reaches a number corresponding to the first threshold length, the embodiment of the present invention obtains a pair of grayscale frame images from the grayscale frame array to perform a difference operation to obtain a residual frame image. In this embodiment of the present invention, the first grayscale frame image includes a first frame image and a second frame image, and the first frame image is the previous frame image of the second frame image. Accordingly, when the pixel value of the residual image meets a preset pixel condition, the embodiment of the present invention stores the residual frame image in the second image array to alleviate noise problems caused by large gaps in image continuity, thereby constructing a residual frame array. Furthermore, when it is determined that the array length of the residual frame array is greater than the second threshold length, the embodiment of the present invention performs calculations and analysis based on the pixel points of each residual frame image in the residual frame array to obtain the coordinates of the target pixel points, and then performs preset clustering processing based on the target pixel point coordinates, thereby obtaining a target recognition result, and achieving more accurate night scene target detection. It is easy to understand that the embodiment of the present invention effectively alleviates the influence of video noise, improves the positioning accuracy of the identified target, and thus effectively improves the accuracy of live target detection in night scenes by subtracting the corresponding first frame image and second frame image based on the motion image of the living target in the video to be detected, and confirms the position of the living target object through multiple adjacent residual frame information. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] FIG1 is a flow chart of a method for detecting a target in a night scene according to an embodiment of the present invention;

[0050] FIG2 is a schematic diagram of a residual frame array construction process according to an embodiment of the present invention;

[0051] 3 is a schematic diagram of a process for obtaining target pixel coordinates by performing calculation and analysis based on pixel points of each residual frame image in a residual frame array according to an embodiment of the present invention;

[0052] FIG4 is a schematic diagram of a process for determining target pixel coordinates based on a pixel matrix according to an embodiment of the present invention;

[0053] 5 is a schematic diagram of a process for performing a preset clustering process according to target pixel coordinates to obtain a target recognition result according to an embodiment of the present invention;

[0054] 6 is a schematic diagram of a process for obtaining a target recognition result by predicting based on an adjacency table using a preset priority algorithm according to an embodiment of the present invention;

[0055] 7 is a schematic diagram of a process for processing pixel values ​​in a preset display area according to an embodiment of the present invention;

[0056] FIG8 is a schematic diagram of the overall process of a night scene target detection method provided by an embodiment of the present invention;

[0057] FIG9 is a modular schematic diagram of a night scene target detection system according to an embodiment of the present invention;

[0058] FIG10 is a block diagram showing the principles of a nighttime scene target detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The embodiments described in the embodiments of this application should not be regarded as limitations of this application. All other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0060] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0062] Before introducing the embodiments of the present application, the relevant terms involved in the present application are first explained.

[0063] Deep learning: This technology uses multi-layer neural network models to simulate the neural network structure of the human brain to learn and analyze data. The core idea of ​​deep learning is to extract high-level features from data through multi-level nonlinear transformations and learning, and then perform tasks such as data classification, recognition, and prediction. Its main feature is its multi-layered neural network structure, where each layer learns a set of feature representations that gradually extract high-level features from the data.

[0064] Depth-First Search (DFS) is an algorithm used to traverse or search a tree or graph. The basic idea is to start from a node, follow a path to the deepest node, then backtrack to the previous node and continue exploring other paths until all nodes are traversed.

[0065] With the development and advancement of video analysis and artificial intelligence technologies, more and more events can be detected and identified in real time through video surveillance. For example, in the kitchen monitoring of catering establishments, video analysis can detect rats in the surveillance scene, record their tracks, detect them in advance, and issue an alarm. Rats entering the detection area can be identified. If a rat is detected, an alarm can be issued and the alarm signal can be simultaneously pushed to management personnel, greatly improving the hygiene and safety level of the kitchen. However, in related technologies, in nighttime scenes, due to the accuracy of the detection algorithm, objects in the picture are often misidentified as the target, resulting in false alarms. In addition, the accuracy of deep learning target detection methods relies on large datasets. However, datasets for detecting live targets at night are often difficult to collect, such as datasets of rat activity in nighttime scenes. Searching for them in relevant surveillance videos is time-consuming and labor-intensive. Therefore, the above technical issues need to be urgently addressed.

[0066] One embodiment of the present invention provides a method, system, and storage medium for nighttime target detection, which can effectively improve the accuracy of live target detection in nighttime scenes and enhance the accuracy of target location. Referring to FIG1 , the method of this embodiment of the present invention includes, but is not limited to, steps S110, S120, S130, S140, S150, S160, and S170.

[0067] Specifically, the application process of the method of the embodiment of the present invention includes but is not limited to the following steps:

[0068] S110: Performing preset decoding on the video to be detected to obtain a video frame image.

[0069] S120: Perform grayscale conversion on the video frame image and store it in a first image array to obtain a grayscale frame array.

[0070] S130: When it is determined that the array length of the grayscale frame array is greater than the first threshold length, a grayscale frame image pair is obtained from the grayscale frame array, wherein the grayscale frame image pair includes a first frame image and a second frame image, and the first frame image is a previous frame image of the second frame image.

[0071] S140: Perform a difference operation on the first frame image and the second frame image to obtain a residual frame image.

[0072] S150: When the pixel value of the residual frame image meets the preset pixel condition, the residual frame image is stored in the second image array to obtain a residual frame array.

[0073] S160: When it is determined that the array length of the residual frame array is greater than the second threshold length, performing calculation and analysis based on the pixel points of each residual frame image in the residual frame array to obtain the target pixel coordinate point.

[0074] S170: Performing preset clustering processing according to the target pixel coordinate points to obtain a target recognition result.

[0075] During the operation of this specific embodiment, the present invention first performs a preset decoding on the video to be detected to obtain a video frame image. Specifically, the video to be detected in this embodiment is acquired by a monitoring device or a corresponding video acquisition device. Accordingly, the preset decoding in this embodiment includes frame extraction decoding, such as timed frame extraction decoding and key frame extraction decoding. This involves reading the video to be detected, extracting the frames to be decoded according to a pre-set frame extraction rule, and then decoding the frames using a decoder to obtain the video frame image. Next, the present invention performs grayscale conversion on the video frame image and stores it in a first image array to construct a grayscale frame array. Specifically, grayscale conversion in this embodiment refers to converting the video frame image into a grayscale image. It will be readily understood that by converting the video frame image into a grayscale frame image, the present invention removes color information from the image, simplifies the image, and better displays details and textures in the image, facilitating subsequent target recognition analysis and processing. Accordingly, the present invention can perform grayscale conversion on the video frame image using methods such as average, maximum, and weighted average. Next, the present invention determines whether the array length of the grayscale frame array is greater than a first threshold length. When it is determined that the array length of the grayscale frame array is greater than the first threshold length, an embodiment of the present invention obtains a grayscale frame image pair from the grayscale frame data. Specifically, the first threshold length in the embodiment of the present invention is a pre-set grayscale frame array length threshold, which can be customized. Accordingly, in the embodiment of the present invention, the grayscale frame image pair includes a first frame image and a second frame image, and the first frame image is the previous frame image of the second frame image, that is, the grayscale frame image pair refers to two adjacent frame images. Exemplarily, when the array length of the grayscale frame array is greater than 2, it means that two frames of grayscale frame images are stored in the grayscale frame array. At this time, the embodiment of the present invention obtains the corresponding grayscale frame image pair from the grayscale frame array. Next, the embodiment of the present invention performs a difference operation based on the first frame image and the second frame image to obtain a residual frame image. Specifically, the embodiment of the present invention constructs the corresponding residual frame image by taking the difference between the grayscale frame images of the two frames before and after. It is easy to understand that the embodiment of the present invention can detect moving objects, that is, living targets, in the video to be detected by comparing the differences between the previous and next frames and recording the residual frame image. If the pixel value in the residual image is large, it means that there is obvious movement at the pixel position. Then, by extracting the moving object in the residual frame image, living target tracking can be performed to achieve target tracking and positioning.

[0076] Furthermore, in an embodiment of the present invention, when the pixel values ​​of the residual frame image meet a preset pixel condition, the residual frame image is stored in a second image array to obtain a residual frame array. Specifically, after performing a difference operation on the first and second frames to obtain the residual frame image, the embodiment of the present invention analyzes the pixel values ​​in the residual frame image to determine whether the pixel values ​​meet the preset pixel condition. It is readily understood that in order to mitigate the impact of noise caused by large disparities in image continuity, the embodiment of the present invention determines whether the pixel values ​​of the residual frame image meet the preset pixel condition. When it is determined that the pixel values ​​of the residual frame image meet the preset pixel condition, the embodiment of the present invention stores the residual frame image in the second image array to construct a residual frame array. Conversely, when it is determined that the pixel values ​​of the residual frame image do not meet the preset pixel condition, the embodiment of the present invention discards the residual frame image. Next, the embodiment of the present invention determines whether the residual frame images stored in the residual frame array meet a certain number, that is, whether the array length of the residual frame array is greater than a second threshold length. Specifically, the second threshold length in the embodiment of the present invention is also a preset residual frame array length threshold and can be customized. Accordingly, when it is determined that the array length of the residual frame array is greater than the second threshold length, that is, a sufficient number of residual frame images are stored in the residual frame array, the embodiment of the present invention performs calculation analysis based on the pixel points of each residual frame image in the residual frame array, and analyzes the activity of the living target by analyzing the pixel point data of each residual frame image to obtain the target pixel coordinate point. Finally, the embodiment of the present invention performs preset clustering processing based on the target pixel coordinate point to obtain the target recognition result. The embodiment of the present invention locates the position information of the corresponding living target by clustering the target pixel coordinate point, and obtains the target recognition result, which effectively improves the accuracy of living target detection in night scenes and improves the positioning accuracy of the identified target. It is easy to understand that the embodiment of the present invention has a high detection accuracy in night scenes, and therefore can also be used to improve the efficiency of collecting living target image data sets in night detection.

[0077] 2 , in some embodiments of the present invention, when the pixel value of the residual frame image meets the preset pixel condition, the residual frame image is stored in the second image array to obtain a residual frame array, including but not limited to:

[0078] S210: Binarize the residual frame image to obtain a binary frame image.

[0079] S220: Calculate the pixel mean of the binary frame image to obtain pixel mean data.

[0080] S230: When it is determined that the pixel mean data is less than the preset pixel threshold, the residual frame image is stored in the second image array to obtain a residual frame array.

[0081] In this specific embodiment, the embodiment of the present invention performs a binarization process on the residual frame image to obtain a binary frame image, and then performs a pixel mean calculation on the binary frame image to obtain pixel mean data. Then, when it is determined that the pixel mean data is less than a preset pixel threshold, the embodiment of the present invention stores the corresponding residual frame image in a second image array to construct a residual frame array. Specifically, the binarization process in the embodiment of the present invention refers to limiting the pixel values ​​of the residual frame image to two fixed values, such as 0 and 255, that is, black and white, so as to separate the target from the background and highlight the morphological features of the target. The embodiment of the present invention first performs a binarization process on the residual frame image, and then calculates the corresponding pixel mean to obtain the pixel mean data of each binary frame image. Accordingly, the embodiment of the present invention compares the pixel value of each residual frame image with the preset pixel threshold to determine whether the pixel mean data is less than the preset pixel threshold. For example, an embodiment of the present invention performs binarization processing on the residual frame image to obtain a binary frame image. For example, the binarization threshold is 20. When the pixel value of a pixel in the residual frame image is less than or equal to 20, the pixel value of the corresponding pixel is set to 0. Conversely, when the pixel value of a pixel in the residual frame image is greater than 20, the pixel value of the corresponding pixel is set to 1. Next, the embodiment of the present invention calculates the pixel mean of the binary frame image. Accordingly, when the pixel mean data is greater than a set threshold h, i.e., a preset pixel threshold, the embodiment of the present invention discards the residual frame image to avoid the noise caused by frame loss and the large difference in image continuity between the key frame and the previous frame in common video codec algorithms, and returns to the step of performing the preset decoding of the video to be detected. In addition, when the pixel mean data is less than or equal to the set threshold h, the embodiment of the present invention first stores the residual frame image in a second image array to obtain a residual frame array, and then proceeds to the next step. It is easy to understand that in the residual frame image stored in the second image array in the embodiment of the present invention, the area with non-zero pixel values ​​can be regarded as an area where a living target may appear. In addition, the embodiment of the present invention can further clarify the target position by comparing multiple residual frames.

[0082] 3 , in some embodiments of the present invention, performing calculation and analysis based on the pixel points of each residual frame image in the residual frame array to obtain the target pixel coordinate point includes but is not limited to the following steps:

[0083] S310: Obtain a preset number of residual frame images from the residual frame array to obtain a residual image group.

[0084] S320: performing an AND operation on the corresponding pixel points of each residual frame image in the residual image group to obtain a pixel matrix.

[0085] S330: Determine the target pixel coordinate point according to the pixel matrix.

[0086] In this specific embodiment, the present invention first obtains a preset number of residual frame images from the residual frame array to form a residual image group. Specifically, after determining that the number of residual frame images stored in the residual frame array reaches a preset number of images, that is, the digital length of the residual frame array is greater than a second threshold length, the present invention obtains a preset number of residual frame images from the residual frame data for analysis to obtain target pixel coordinates. The preset number in the present embodiment can be customized, such as being set to the number of images corresponding to the second threshold length. Next, the present embodiment performs an AND operation on the corresponding pixels in each residual frame image in the residual image group to obtain a corresponding pixel matrix. Specifically, the AND operation in the present embodiment is a logical operation in a computer. The two data involved in the operation are ANDed based on binary bits. The result is 1 only when both bits are 1; otherwise, it is 0. The present embodiment performs an AND operation on the corresponding pixels in each residual frame image to construct a pixel matrix reflecting the activity state of the living target. Furthermore, the embodiment of the present invention determines the target pixel coordinates based on the pixel matrix, and determines whether there is a corresponding living target and the corresponding pixel coordinates in the image by analyzing the pixel points of each residual frame image and the pixel matrix obtained by calculation. For example, the embodiment of the present invention takes out the residual frames c of the first o frames in the residual frame array. i 、c i-1 、c i-2 ,…,c i-o+1 , and obtain the residual image group. Then, the embodiment of the present invention performs an AND operation on each pixel point corresponding to each residual frame image in the residual image group: r=c i &c i-1 &c i-2 &…&c i-o+1 Then, the embodiment of the present invention analyzes the pixel matrix to determine the corresponding target pixel coordinates.

[0087] 3 and 4 , in some embodiments of the present invention, determining the target pixel coordinate point according to the pixel matrix includes but is not limited to the following steps:

[0088] S410: When it is determined that the element values ​​of the pixel matrix are not all zero, the pixel coordinates corresponding to the element values ​​that are not zero in the pixel matrix are obtained to obtain the target pixel coordinate point.

[0089] In this specific embodiment, the present invention analyzes the element values ​​in the pixel matrix to determine the corresponding target pixel coordinates. Specifically, the present invention performs an AND operation on each pixel corresponding to each residual frame image in the residual image group. The result is 1 only if both bits in the AND operation are simultaneously 1; otherwise, it is 0. Therefore, the present invention determines whether a living object is moving in the corresponding image frame by determining whether the element values ​​in the pixel matrix are all 0. Accordingly, if the values ​​in the matrix r are all 0, it is assumed that there is no living object, such as a mouse, in the frame. Conversely, if the values ​​are not all 0, it is assumed that there is living object activity in the non-zero area. It is easy to understand that the residual frame image obtained by subtracting the previous and next frame images in the present embodiment represents the difference between the previous and next frames. When a living object moves, it typically causes changes in the pixels in the image. Therefore, the present invention performs an AND operation on the corresponding pixels in the residual frame images to binarize the difference between the two frames, thereby converting changed pixels to 1 and unchanged pixels to 0. Therefore, if there is a living target activity in the image, there will be a large number of 1s in the result after the AND operation, otherwise there will be fewer 1s. Furthermore, the embodiment of the present invention can determine whether there is living activity in the image through the pixel matrix obtained by the AND operation. Accordingly, when it is determined that the element values ​​in the pixel matrix are not all zero, the embodiment of the present invention believes that there is a living target in the corresponding image, and the embodiment of the present invention obtains the pixel coordinates corresponding to the non-zero element values ​​in the pixel matrix to obtain the target pixel coordinate point. Conversely, when it is determined that the element values ​​in the pixel matrix are all zero, it is considered that there is no living target in the corresponding image, and the corresponding image frame is discarded.

[0090] 5 , in some embodiments of the present invention, a preset clustering process is performed based on the target pixel coordinates to obtain a target recognition result, including but not limited to the following steps:

[0091] S510: searching for contour points of a non-zero connectivity area according to target pixel coordinates using a preset search algorithm to obtain several groups of target contour points.

[0092] S520: Calculate the coordinate mean of each group of target contour points to obtain the corresponding approximate center point.

[0093] S530: Calculate the Euclidean distance between the approximate center points to obtain center distance data.

[0094] S540: When it is determined that the center distance data is less than the preset distance threshold, an adjacency table is established.

[0095] S550: Analyze the adjacency list using a preset priority algorithm to obtain a target recognition result.

[0096] In this specific embodiment, the present invention first uses a preset search algorithm to search for contour points of non-zero connected regions based on target pixel coordinates, thereby obtaining several groups of target contour points. Specifically, the preset search algorithm in the present invention refers to an algorithm for detecting contours in an image. Accordingly, the preset search algorithms in the present invention include the findContours algorithm, the Cannny edge detection algorithm, and the Sobel operator. The present invention uses the preset search algorithm to search for contour points of non-zero connected regions, i.e., to search for contour points of regions connected to pixels whose calculated result is 1. Next, the present invention calculates the coordinate mean of each group of target contour points to obtain corresponding approximate center points, and then calculates the Euclidean distance between the approximate center points to obtain center distance data. Specifically, after calculating several groups of target contour points, the present invention calculates the coordinate mean of each group of target contour points to obtain a corresponding number of approximate center points of non-zero connected regions. Then, the present invention calculates the Euclidean distance between each pair of approximate center points to obtain corresponding center distance data. Furthermore, the present invention determines whether the center distance data is less than a preset distance threshold, i.e., whether the distance between the approximate center points is less than the preset distance threshold. When it is determined that the center distance data is less than a preset distance threshold, an embodiment of the present invention establishes an adjacency table. It is easy to understand that when the center distance data is less than the preset distance threshold, that is, the Euclidean distance between the approximate center points is less than the preset threshold, then they are considered to be in the same target area, and an embodiment of the present invention establishes a corresponding adjacency table. Among them, the adjacency table in the embodiment of the present invention is a data structure for representing the relationship between approximate center points, which can reflect the relationship between target areas. Accordingly, when establishing the adjacency table, each approximate center point can be regarded as a vertex. When two approximate center points are judged to be in the same area, that is, the center distance data is less than the preset distance threshold, an edge is added to the adjacency table to represent the relationship between them, so that the adjacency table can record the connection between each approximate center point and other approximate center points. Finally, the embodiment of the present invention analyzes the adjacency table through a preset priority algorithm to obtain a target recognition result. Specifically, the preset priority algorithm in the embodiment of the present invention includes a depth-first search (DFS) or a breadth-first search (BFS). The embodiment of the present invention analyzes the adjacency list through a preset priority algorithm to group the approximate center points of the same target area, thereby obtaining a target recognition result.

[0097] 6 , in some embodiments of the present invention, analyzing the adjacency table using a preset priority algorithm to obtain a target recognition result includes but is not limited to the following steps:

[0098] S610: Traverse the adjacency list using a depth-first algorithm, group approximate center points, and obtain center point group data.

[0099] S620: Group the target contour points according to the center point grouping data and calculate the corresponding corner points to obtain corner point data.

[0100] S630: Calculate the circumscribed rectangle based on the corner point data to obtain the target recognition result.

[0101] In this specific embodiment, the embodiment of the present invention first traverses the adjacency list through the depth-first algorithm to group the approximate center points to obtain corresponding center point grouping data, and then groups the contour points according to the center point grouping data and calculates the corresponding corner points to obtain corner point data, thereby obtaining the target recognition result by finding the circumscribed rectangle of the corner point data. Specifically, the embodiment of the present invention traverses the adjacency list through the depth-first algorithm (DFS) to group the approximate center points that are considered to be the same target area to obtain several groups of approximate center points, namely, the corresponding center point grouping data. Accordingly, the embodiment of the present invention groups the corresponding several groups of contour points according to the groups of their corresponding center points to obtain several groups of contour points. Further, the embodiment of the present invention calculates the corresponding corner points according to the grouping of the contour points to obtain the corresponding corner point data, and respectively calculates the circumscribed rectangles of these grouped corner points to obtain the recognized rectangle coordinates, namely, the target recognition result. It should be noted that the embodiment of the present invention sets a threshold based on the non-zero connected area obtained by image processing and the volume characteristics of the detection target, groups adjacent and similar connected areas into a group, and then generates a circumscribed rectangular target frame, which has a more accurate control over the overall detection target. The target frame can more accurately locate the overall correct coordinates of the detection target, alleviating the problem that multiple small target frames may be detected for the same detection target as a whole.

[0102] In conjunction with FIG1 and FIG7 , in some embodiments of the present invention, a method for detecting a target in a night scene provided by an embodiment of the present invention further includes but is not limited to the following steps:

[0103] S710: Setting the pixel value of the preset display area of ​​each grayscale image in the grayscale frame array to 0. The preset display area includes a time display area.

[0104] In this specific embodiment, after the video frame image is grayscale converted, the embodiment of the present invention also sets the pixel values ​​of the preset display areas in each grayscale image to 0. Specifically, the preset display area in the embodiment of the present invention includes a time display area. It is easy to understand that the video to be detected acquired by the embodiment of the present invention through the monitoring device or the corresponding video acquisition device will have a time display, that is, there is a time display area in the video frame image. However, since the time display area will jump over time, it is easy to cause recognition interference. Therefore, the embodiment of the present invention alleviates the interference caused by the jumping of the time display area in the picture by setting the pixel values ​​of the time display area of ​​each grayscale image in the grayscale frame array to 0. It should be noted that in some embodiments of the present invention, the preset display area may also include other text display areas in the video to be detected. By setting the pixel values ​​of the preset display area to 0, the interference caused by non-environmental reasons is reduced, thereby improving the accuracy of target detection.

[0105] Referring to FIG8 , FIG8 is a schematic diagram of the overall process of the night scene target detection method provided by an embodiment of the present invention. For example, the night scene target detection method is applied to the rat identification scene under the infrared mode of the night kitchen camera. The embodiment of the present invention first acquires the video to be detected by the monitoring device or the corresponding video acquisition device. Then, the embodiment of the present invention performs a preset decoding on the video to be detected to obtain a video frame image. For example, the embodiment of the present invention obtains the video frame image t by performing frame extraction decoding on the video to be detected. i Then, the embodiment of the present invention performs grayscale conversion on the video frame image, sets the pixel value of the corresponding time display area to 0, and stores it in the first image array, thereby constructing a grayscale frame array q1. Furthermore, the embodiment of the present invention determines whether the array length of the grayscale frame array is greater than the first threshold length, such as the grayscale frame array q i When it is determined that the array length of the grayscale frame array q1 is greater than the first threshold length, the embodiment of the present invention obtains the corresponding grayscale frame image pair from the grayscale frame array q1, including the first frame image g i-1 and the second frame image g i , and the first frame image g i-1 The second frame image g i Accordingly, the embodiment of the present invention is based on the first frame image g i-1 and the second frame image g i Perform a difference operation to obtain the corresponding residual frame image c i Then, the embodiment of the present invention converts the residual frame image c iA median blurring process is performed to remove some noise. At the same time, the embodiment of the present invention performs a binarization process. For example, if the threshold is 20, the pixel value is set to 0 if it is less than or equal to 20, and the pixel value is set to 1 if it is greater than 20, thereby obtaining a corresponding binary frame image. Furthermore, the embodiment of the present invention performs a pixel mean calculation on the binary frame image to obtain pixel mean data, and determines whether the pixel mean data is less than or equal to a preset pixel threshold. For example, in the embodiment of the present invention, when the calculated pixel mean data is greater than the set threshold h, the corresponding residual frame image is discarded to alleviate the noise impact caused by frame loss and the large gap in image continuity between the key frame and the previous frame of the common video encoding and decoding algorithm. On the contrary, when it is determined that the calculated pixel mean data is less than the set threshold h, the embodiment of the present invention stores the corresponding residual frame image in the second image array to construct a residual frame array q2.

[0106] Furthermore, the embodiment of the present invention determines whether the array length of the residual frame array q2 is greater than a second threshold length, such as whether the array length of the residual frame array q2 is greater than x. When the array length of the residual frame array q2 is less than x, the embodiment of the present invention repeats the above steps until the number of residual frame images stored in the residual frame array q2 reaches a preset number of images. Accordingly, when it is determined that the array length of the residual frame array is greater than the second threshold length, the embodiment of the present invention obtains a preset number of residual frame images from the residual frame array q2 to form a residual image group, and performs an AND operation on the corresponding pixel points of each residual frame image in the residual image group to obtain a pixel matrix. For example, the present invention takes out the residual frame image c of the first o frames in the residual frame array q2. i 、c i-1 、c i-2 ,…,c i-o+1 , forming a residual image group. Then, the embodiment of the present invention performs an AND operation on each pixel point corresponding to each residual frame image in the residual image group: r=c i &c i-1 &c i-2 &…&c i-o+1, and obtain a pixel matrix. Next, the embodiment of the present invention determines whether the elements in the pixel matrix are all zero. If the values ​​of the matrix r are all zero, it is considered that there is no mouse in the frame. Conversely, if they are not all zero, it is considered that there is mouse activity in the non-zero area, thereby obtaining the target pixel coordinate point, and then clustering the r non-zero value coordinate points. Accordingly, the embodiment of the present invention searches for the contour points of the non-zero connected area based on the target pixel coordinate point through a preset search algorithm to obtain several groups of target contour points, and then calculates the coordinate mean of each group of target contour points to obtain the corresponding approximate center point. By calculating the Euclidean distance between the approximate center points, the center distance data is obtained, and an adjacency list is constructed based on the approximate center points whose center distance data is less than the preset distance threshold. For example, an embodiment of the present invention uses the findContours method to search for non-zero connected areas in the pixel matrix r, obtaining n groups of target contour points. The coordinate mean of each of these n groups of target contour points is then calculated to obtain the approximate center points of the n corresponding non-zero connected areas. The Euclidean distance between the n approximate center points is then calculated. When the Euclidean distance is less than a threshold j, the approximate center points are considered to be the same target area, thereby establishing a corresponding adjacency table. Furthermore, an embodiment of the present invention uses a depth-first algorithm to traverse the adjacency table to group the approximate center points, obtaining center point grouping data. The target contour points are then grouped based on the center point grouping data, and the corresponding corner points are calculated to obtain corner point data. The bounding rectangle is then calculated based on the corner point data to obtain the target recognition result. For example, an embodiment of the present invention uses a depth-first algorithm (DFS) to traverse the adjacency table, grouping the approximate center points that are considered to be the same target area, obtaining k groups of approximate center points. Furthermore, an embodiment of the present invention groups the n groups of target contour points based on the corresponding approximate center point groups, thereby obtaining k groups of target contour points. Then, the embodiment of the present invention calculates the bounding rectangles of the k groups of corner points respectively, and the obtained rectangle coordinates are the coordinates of the detected mouse. i And the obtained rectangle coordinate information is the target recognition result.

[0107] 9 , an embodiment of the present invention further provides a nighttime scene target detection system, comprising:

[0108] The first module 810 is configured to perform preset decoding on the video to be detected to obtain a video frame image.

[0109] The second module 820 is configured to perform grayscale conversion on the video frame image and store the grayscale image in the first image array to obtain a grayscale frame array.

[0110] The third module 830 is configured to obtain a grayscale frame image pair from the grayscale frame array when it is determined that the array length of the grayscale frame array is greater than the first threshold length. The grayscale frame image pair includes a first frame image and a second frame image, where the first frame image is a previous frame image of the second frame image.

[0111] The fourth module 840 is configured to perform a difference operation on the first frame image and the second frame image to obtain a residual frame image.

[0112] The fifth module 850 is configured to store the residual frame image in the second image array to obtain a residual frame array when the pixel value of the residual frame image meets a preset pixel condition.

[0113] The sixth module 860 is configured to, when it is determined that the array length of the residual frame array is greater than the second threshold length, perform calculation and analysis based on the pixel points of each residual frame image in the residual frame array to obtain the target pixel coordinate point.

[0114] The seventh module 870 is used to perform preset clustering processing according to the target pixel coordinate points to obtain a target recognition result.

[0115] The contents of the method embodiments of the present invention are all applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0116] 10 , an embodiment of the present invention further provides a nighttime scene target detection system, comprising:

[0117] At least one processor 910 .

[0118] At least one memory 920 is configured to store at least one program.

[0119] When the at least one program is executed by the at least one processor 910, the at least one processor 910 implements the night scene target detection method as described in the above embodiment.

[0120] The contents of the method embodiments of the present invention are all applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0121] An embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, which are executed by one or more control processors, for example, to execute the steps of the night scene target detection method described in the above embodiment.

[0122] The contents of the method embodiments of the present invention are all applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0123] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0124] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0126] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0127] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0128] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0129] The step numbers in the above method embodiment are only provided for the convenience of explanation and do not limit the order of the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0130] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A night scene target detection method, characterized in that: The following steps are involved: Perform preset decoding on the video to be detected to obtain a video frame image; Convert the video frame image into grayscale and store it in a first image array to obtain a grayscale frame array; When it is determined that the array length of the grayscale frame array is greater than the first threshold length, a grayscale frame image pair is obtained from the grayscale frame array; wherein the grayscale frame image pair includes a first frame image and a second frame image, and the first frame image is a previous frame image of the second frame image; Performing a difference operation on the first frame image and the second frame image to obtain a residual frame image; When the pixel value of the residual frame image meets the preset pixel condition, the residual frame image is stored in the second image array to obtain a residual frame array; When it is determined that the array length of the residual frame array is greater than the second threshold length, performing calculation and analysis according to the pixel points of each residual frame image in the residual frame array to obtain the target pixel coordinate point; A preset clustering process is performed according to the target pixel coordinate points to obtain a target recognition result.

2. The night scene target detection method according to claim 1, characterized in that: When the pixel value of the residual frame image meets the preset pixel condition, the residual frame image is stored in the second image array to obtain a residual frame array, including: Binarizing the residual frame image to obtain a binary frame image; Calculating the pixel mean of the binary frame image to obtain pixel mean data; When it is determined that the pixel mean data is less than a preset pixel threshold, the residual frame image is stored in the second image array to obtain the residual frame array.

3. The night scene target detection method according to claim 1, characterized in that: The step of performing calculation and analysis according to the pixel points of each residual frame image in the residual frame array to obtain the target pixel coordinate point comprises: Acquire a preset number of the residual frame images from the residual frame array to obtain a residual image group; Performing an AND operation on the corresponding pixel points of each residual frame image in the residual image group to obtain a pixel matrix; The target pixel coordinate point is determined according to the pixel matrix.

4. The night scene target detection method according to claim 3, characterized in that: The step of determining the target pixel coordinate point according to the pixel matrix includes: When it is determined that the element values ​​of the pixel matrix are not all zero, the pixel coordinates corresponding to the element values ​​that are not zero in the pixel matrix are obtained to obtain the target pixel coordinate point.

5. The night scene target detection method according to claim 1, characterized in that: The performing of a preset clustering process according to the target pixel coordinate points to obtain a target recognition result includes: According to the target pixel coordinate points, a preset search algorithm is used to find contour points of the non-zero connected area to obtain a plurality of groups of target contour points; Calculating the coordinate mean of each group of target contour points to obtain the corresponding approximate center point; Calculating the Euclidean distance between the approximate center points to obtain center distance data; When it is determined that the center distance data is less than a preset distance threshold, an adjacency table is established; The target recognition result is obtained by analyzing the adjacency list through a preset priority algorithm.

6. The night scene target detection method according to claim 5, characterized in that: The target recognition result is obtained by analyzing the adjacency table through a preset priority algorithm, including: Traversing the adjacency list by using a depth-first algorithm, grouping the approximate center points, and obtaining center point grouping data; The target contour points are grouped according to the center point grouping data and corresponding corner points are calculated to obtain corner point data; The circumscribed rectangle is calculated based on the corner point data to obtain the target recognition result.

7. The night scene target detection method according to claim 1, characterized in that: The method further comprises: The pixel value of the preset display area of ​​each grayscale image in the grayscale frame array is set to 0; wherein the preset display area includes a time display area.

8. A night scene target detection system, characterized in that: include: The first module is used to perform preset decoding on the video to be detected to obtain a video frame image; The second module is used to convert the video frame image into grayscale and store it in the first image array to obtain a grayscale frame array; A third module is configured to obtain a grayscale frame image pair from the grayscale frame array when it is determined that the array length of the grayscale frame array is greater than a first threshold length; wherein the grayscale frame image pair includes a first frame image and a second frame image, and the first frame image is a previous frame image of the second frame image; A fourth module is used to perform a difference operation based on the first frame image and the second frame image to obtain a residual frame image; A fifth module is used for storing the residual frame image in a second image array to obtain a residual frame array when the pixel value of the residual frame image meets a preset pixel condition; A sixth module is used for, when it is determined that the array length of the residual frame array is greater than a second threshold length, performing calculation and analysis according to the pixel points of each residual frame image in the residual frame array to obtain a target pixel coordinate point; The seventh module is used to perform preset clustering processing according to the target pixel coordinate points to obtain a target recognition result.

9. A night scene target detection system, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the night scene target detection method as described in any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the night scene target detection method as described in any one of claims 1 to 7 when executed by the processor.

Images