Security video monitoring fault detection method and device, terminal equipment and storage medium

Abstract

The invention discloses a security and protection video monitoring fault detection method and device, terminal equipment and a storage medium, and the method comprises the steps: determining the similarity between a first video frame collected at a previous detection moment and a second video frame collected at a current detection moment is lower than a first threshold value; detecting whether the monitoring equipment receives a position adjusting instruction or not in an interval time period between the previous detection moment and the current detection moment; if not, determining that the monitoring equipment has an image acquisition fault; if yes, determining a theoretical monitoring position which should be set by the monitoring equipment at each moment in the interval time period according to the position adjustment instruction; generating a theoretical image path according to each theoretical monitoring position; generating an actual image path according to an actual video frame actually acquired by the monitoring equipment; and if the theoretical image path is consistent with the actual image path, determining that the monitoring equipment does not have a fault, otherwise, determining that the monitoring equipment has a movement fault. According to the invention, the accuracy of fault detection of the monitoring equipment can be improved.

Description

Technical Field

[0001] The present invention relates to the field of security monitoring technology, and in particular to a security video monitoring fault detection method, device, terminal equipment and storage medium. Background Art

[0002] Monitoring equipment can be used for remote monitoring, which plays a significant role in the safety protection of people's daily life. For monitoring equipment fault monitoring, the judgment is generally based on the images collected by the monitoring equipment.

[0003] In the prior art, generally, the similarity of video frames at two acquisition time points is compared to determine whether there is a sudden picture change between the two video frames. If there is a sudden picture change, it means that a fault has occurred.

[0004] However, the above fault judgment method is too simple. In actual scenarios, a monitoring device often needs to monitor multiple areas. For this reason, it is sometimes necessary to control the movement of the monitoring device in a timely manner to monitor other areas. The movement of the monitoring device will inevitably lead to sudden changes in the monitoring screen. Using the above fault judgment method, it is easy to judge that the monitoring device has a fault after this normal scheduling operation is executed, resulting in misjudgment. Summary of the invention

[0005] The present invention provides a security video monitoring fault detection method, device, terminal equipment and storage medium capable of improving the accuracy of monitoring equipment fault judgment.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a security video surveillance fault detection method, comprising:

[0007] Acquire a first video frame captured by the monitoring device at a previous detection moment and a second video frame captured at a current detection moment; if it is determined that a first similarity between the first video frame and the second video frame is less than a first threshold, detect whether the monitoring device receives a position adjustment instruction during an interval between the previous detection moment and the current detection moment;

[0008] If not, it is determined that the monitoring device has an image acquisition failure;

[0009] If so, the theoretical monitoring position that the monitoring device should be set at each moment in the interval period is determined according to the position adjustment instruction; according to each theoretical monitoring position, the video frame at the corresponding position is identified from the preset spliced ​​image to obtain each first standard video frame; the center point of each first standard video frame is extracted, and according to the moment corresponding to the theoretical monitoring position, the center points of each first standard video are connected in chronological order to generate the theoretical image path that the monitoring device should correspond to when moving according to the position adjustment instruction; wherein, the video frames shot by the monitoring device at all monitoring positions in the historical period are spliced ​​according to the corresponding monitoring positions to generate a spliced ​​image;

[0010] Calculate the similarity between each actual video frame actually collected by the monitoring device and each video frame in the spliced ​​image at each moment of the interval period, use the video frame with the highest similarity in the spliced ​​image as the second standard video frame corresponding to the actual video frame, extract the center point of each second standard video frame, connect the center points of each second standard video frame in chronological order, and generate the actual image path corresponding to the actual movement of the monitoring device;

[0011] It is determined whether the theoretical image path and the actual image path are consistent. If they are consistent, it is determined that the monitoring device has no fault. If they are inconsistent, it is determined that the monitoring device has a movement fault.

[0012] Further, the similarity between the first video frame and the second video frame is calculated in the following manner:

[0013] Determine a static image area and a dynamic image area of ​​a first video frame;

[0014] A three-dimensional coordinate system is constructed with the row coordinates of the pixel point in the image as the X-axis, the column coordinates of the pixel point in the image as the Y-axis, and the pixel value of the pixel point as the Y-axis;

[0015] Projecting pixel points in the static image area of ​​the first video frame into a three-dimensional coordinate system to generate a first spatial projection;

[0016] Projecting the pixel points in the dynamic image area of ​​the first video frame into a three-dimensional coordinate system to generate a second spatial projection;

[0017] Projecting the pixel points at the corresponding position in the second video frame into a three-dimensional coordinate system according to the static image area of ​​the first video frame to generate a third space projection;

[0018] Projecting the pixel points at the corresponding position in the second video frame into the three-dimensional coordinate system according to the dynamic image area of ​​the first video frame to generate a fourth spatial projection;

[0019] Calculating the similarity between the first spatial projection and the third spatial projection to obtain a static similarity; calculating the similarity between the second spatial projection and the fourth spatial projection to obtain a dynamic similarity;

[0020] The comprehensive similarity is calculated according to the static similarity, the first preset weight corresponding to the static similarity, the dynamic similarity and the second preset weight corresponding to the dynamic similarity; wherein the first preset weight is greater than the second preset weight;

[0021] The comprehensive similarity is used as the similarity between the first video frame and the second video frame.

[0022] Further, the determining of the static image area and the dynamic image area of ​​the first video frame includes:

[0023] Determine the monitoring position of the monitoring device at the previous detection moment according to the first standard image corresponding to the first video frame in the stitched image, and obtain the target monitoring position;

[0024] Obtain each historical video frame collected by the monitoring device at the target monitoring position at each historical moment; generate a historical video group according to each historical video frame; wherein each historical video group includes two historical video frames that are adjacent in time sequence;

[0025] For each historical video group, each historical video frame in the historical video group is divided into a number of blocks, the second similarity of the blocks corresponding to the two historical video frames is calculated one by one, and the area corresponding to the block group whose second similarity is less than the second threshold is used as the dynamic area to be selected corresponding to the historical video group;

[0026] Taking the intersection of the to-be-selected dynamic regions corresponding to all historical video groups, to obtain the dynamic image region of the first video frame;

[0027] All image regions except the dynamic image region in the first video frame are regarded as static image regions.

[0028] Further, when it is determined that the monitoring device has a movement failure, the path point at which the theoretical image path and the actual image path begin to differ is determined according to the overlap between the theoretical image path and the actual image path, and the difference path point is obtained;

[0029] The time and position when the monitoring device has a movement failure are determined according to the time and position corresponding to the video frame where the difference path point is located.

[0030] Further, after determining that the monitoring device has a movement failure, the method further includes:

[0031] Determine the position of the first standard video frame corresponding to the previous detection moment and the current detection moment in the spliced ​​image, and obtain a first position and a second position respectively;

[0032] According to the first position and the second position, determining in the stitched image an image path that avoids the difference path point and is the shortest when traveling from the first position to the second position, to obtain a target image path;

[0033] The monitoring device is controlled to move in sequence according to the monitoring positions corresponding to the target image path to complete the position correction.

[0034] Furthermore, when it is determined that the monitoring device has an image acquisition failure, the method further includes:

[0035] Extracting the RGB value of each pixel in the second video frame;

[0036] If the RGB values ​​of all pixels are all zero, it is determined that the monitoring device has a black screen failure;

[0037] If the RGB values ​​of all pixels are 255, it is determined that the monitoring device has a white screen failure;

[0038] Otherwise, it is determined that the monitoring device has been illegally moved or suffered from an image tampering attack.

[0039] Furthermore, after determining that a black screen failure occurs on the monitoring device, the method further includes:

[0040] According to each actual video frame actually collected by the monitoring device at each moment of the interval period, it is determined whether there is a human object that continuously approaches the monitoring device;

[0041] If so, it is determined that the black screen failure is caused by malicious human blocking; if not, it is determined that the black screen failure is caused by damage to the monitoring equipment.

[0042] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0043] An embodiment of the present invention provides a security video surveillance fault detection device, comprising: an instruction determination module, a first fault identification module, a theoretical image path generation module, an actual image path generation module and a second fault identification module;

[0044] The instruction determination module is used to obtain a first video frame collected by the monitoring device at a previous detection moment and a second video frame collected at a current detection moment; if it is determined that the first similarity between the first video frame and the second video frame is less than a first threshold, detect whether the monitoring device receives a position adjustment instruction during an interval between the previous detection moment and the current detection moment;

[0045] The first fault identification module is used to determine that an image acquisition fault occurs in the monitoring device when it is determined that the monitoring device has not received the position adjustment instruction;

[0046] The theoretical image path generation module is used to determine the theoretical monitoring position that the monitoring device should be set at each moment in the interval period according to the position adjustment instruction when it is determined that the monitoring device has received the position adjustment instruction; identify the video frame at the corresponding position from the preset spliced ​​image according to each theoretical monitoring position, obtain each first standard video frame, extract the center point of each first standard video frame, and connect the center points of each first standard video in chronological order according to the moment corresponding to the theoretical monitoring position, so as to generate the theoretical image path that the monitoring device should correspond to when moving according to the position adjustment instruction; wherein, the video frames shot by the monitoring device at all monitoring positions in the historical period are spliced ​​according to the corresponding monitoring positions to generate a spliced ​​image;

[0047] The actual image path generation module is used to calculate the similarity between each actual video frame actually collected by the monitoring device and each video frame in the spliced ​​image at each time of the interval period, use the video frame with the highest similarity in the spliced ​​image as the second standard video frame corresponding to the actual video frame, extract the center point of each second standard video frame, connect the center points of each second standard video frame in chronological order, and generate the actual image path corresponding to the actual movement of the monitoring device;

[0048] The second fault identification module is used to determine whether the theoretical image path and the actual image path are consistent. If they are consistent, it is determined that the monitoring device has no fault. If they are inconsistent, it is determined that the monitoring device has a movement fault.

[0049] Based on the above method embodiment, the present invention provides a corresponding terminal device embodiment;

[0050] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the security video surveillance fault detection method described in any one of the present invention is implemented.

[0051] Based on the above method embodiment, the present invention provides a storage medium embodiment;

[0052] An embodiment of the present invention provides a storage medium, wherein the storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute any one of the security video surveillance fault detection methods described in the present invention.

[0053] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0054] The embodiment of the present invention provides a security video surveillance fault detection method, device, terminal device and storage medium. When determining that the similarity between the first video frame collected at the previous detection moment and the second video frame collected at the current detection moment is lower than the first threshold, the method detects whether the monitoring device receives a position adjustment instruction during the interval between the previous detection moment and the current detection moment. If not, it is determined that the monitoring device has an image acquisition failure; if so, the theoretical monitoring position that the monitoring device should set at each moment in the interval is determined according to the position adjustment instruction; according to each theoretical monitoring position, the video frame at the corresponding position is identified from the preset spliced ​​image to obtain each first standard video frame; the center point of each first standard video frame is extracted, and according to the moment corresponding to the theoretical monitoring position, the center points of each first standard video are connected in chronological order to generate the theoretical image path that the monitoring device should correspond to when moving according to the position adjustment instruction; wherein, the video frames taken by the monitoring device at all monitoring positions in the historical period are connected. , generate a spliced ​​image after splicing according to the corresponding monitoring position; calculate the similarity between each actual video frame actually collected by the monitoring device and each video frame in the spliced ​​image at each moment of the interval period, use the video frame with the highest similarity in the spliced ​​image as the second standard video frame corresponding to the actual video frame, extract the center point of each second standard video frame, connect the center points of each second standard video in chronological order, and generate the actual image path corresponding to the actual movement of the monitoring device; judge whether the theoretical image path and the actual image path are consistent, if they are consistent, it is determined that the monitoring device has no fault, if they are inconsistent, it is determined that the monitoring device has a moving fault. Compared with the prior art, the present application does not directly determine the monitoring device fault when it is determined that there is a sudden change in the picture, but further introduces the query of the position adjustment instruction to exclude the situation of sudden change in the picture caused by the normal position adjustment of the monitoring device, thereby improving the accuracy of fault judgment. In addition, by comparing the theoretical image path with the actual image path, it is further determined whether the monitoring device has a fault in the position adjustment process. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of a security video monitoring fault detection method provided by one embodiment of the present invention;

[0056] Figure 2 is a schematic diagram of a spliced ​​image provided by an embodiment of the present invention;

[0057] Figure 3 is a schematic diagram of a theoretical image path provided by an embodiment of the present invention;

[0058] Figure 4is a schematic diagram of an actual image path provided by an embodiment of the present invention;

[0059] Figure 5 is a schematic diagram of an actual image path provided by another embodiment of the present invention;

[0060] Figure 6 is a schematic diagram of difference path points provided by an embodiment of the present invention;

[0061] Figure 7 The present invention is a schematic structural diagram of a security video monitoring fault detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0064] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relativity or implicitly indicating the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0065] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0066] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0067] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0068] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0069] Please refer to Figure 1 , is a flow chart of a security video surveillance fault detection method provided by an embodiment of the present invention, comprising the following specific steps:

[0070] S1. Obtain a first video frame collected by the monitoring device at a previous detection moment and a second video frame collected at a current detection moment; when it is determined that a first similarity between the first video frame and the second video frame is less than a first threshold, detect whether the monitoring device receives a position adjustment instruction during an interval between the previous detection moment and the current detection moment;

[0071] Specifically, in the present invention, each detection moment is pre-set according to a preset time period. Schematically, it can be set to perform a detection every 1 minute, and the difference between two adjacent detection moments is 1 minute. After the video frame of the monitoring device, i.e., the above-mentioned second video frame, is obtained at the current detection moment, the video frame obtained at the current detection moment is compared with the video frame collected by the monitoring device at the previous detection moment, i.e., the above-mentioned first video frame, and the similarity between the two is calculated. When the similarity is less than the set first threshold, it is determined that a sudden change has occurred in the monitoring screen at this time. This sudden change may be caused by a fault, or it may be caused by the monitoring device receiving a position adjustment instruction and adjusting its own position according to the adjustment method and adjustment range specified by the position adjustment instruction, resulting in a change in the monitoring area, thereby causing a sudden change in the screen. In order to distinguish these two situations, when the similarity is less than the set first threshold, it is queried whether the monitoring device has received a position adjustment instruction in the interval period from the previous detection moment to the current detection moment. If yes, it means that the sudden change in the screen is caused by the adjustment of the monitoring device to its own position. If not, it means that the sudden change in the screen is caused by a fault.

[0072] In order to more accurately compare the similarity between the first video frame and the second video frame, in a preferred embodiment, the similarity between the first video frame and the second video frame may be calculated in the following manner:

[0073] Determine a static image area and a dynamic image area of ​​a first video frame;

[0074] A three-dimensional coordinate system is constructed with the row coordinates of the pixel point in the image as the X-axis, the column coordinates of the pixel point in the image as the Y-axis, and the pixel value of the pixel point as the Y-axis;

[0075] Projecting pixel points in the static image area of ​​the first video frame into a three-dimensional coordinate system to generate a first spatial projection;

[0076] Projecting the pixel points in the dynamic image area of ​​the first video frame into a three-dimensional coordinate system to generate a second spatial projection;

[0077] Projecting the pixel points at the corresponding position in the second video frame into a three-dimensional coordinate system according to the static image area of ​​the first video frame to generate a third space projection;

[0078] Projecting the pixel points at the corresponding position in the second video frame into the three-dimensional coordinate system according to the dynamic image area of ​​the first video frame to generate a fourth spatial projection;

[0079] Calculating the similarity between the first spatial projection and the third spatial projection to obtain a static similarity; calculating the similarity between the second spatial projection and the fourth spatial projection to obtain a dynamic similarity;

[0080] The comprehensive similarity is calculated according to the static similarity, the first preset weight corresponding to the static similarity, the dynamic similarity and the second preset weight corresponding to the dynamic similarity; wherein the first preset weight is greater than the second preset weight;

[0081] The comprehensive similarity is used as the similarity between the first video frame and the second video frame.

[0082] In actual scenarios, a monitoring screen may have static scenes, such as some buildings, fixed equipment, etc., and dynamic scenes such as the sky. Different weather conditions will cause the corresponding screen area of ​​the sky to change, such as the movement of clouds, or from sunny to cloudy. The pixel values ​​of the pixels corresponding to the static scenes are generally not found to change, but the pixel values ​​of the pixels corresponding to the dynamic scenes will change with the change of time. These changes will cause the similarity between the two images to decrease. In the actual monitoring process, if the difference between the images between the previous and next detection moments is only the difference in the dynamic area, we should generally consider this to be a normal picture change and should not be judged as an image mutation. Therefore, in the actual similarity comparison process, it is necessary to weaken the impact of the dynamic area picture change on the image similarity between the previous and next detection moments;

[0083] To this end, in this embodiment of the present invention, the static image area and the dynamic image area of ​​the first video frame are first determined;

[0084] Then, the pixel points of the static image area of ​​the first video frame are projected into a corresponding three-dimensional coordinate system according to the coordinates and pixel values ​​of the pixel points in the image to generate a first spatial projection;

[0085] Next, pixel points of an area corresponding to the static image area of ​​the first video frame are extracted from the second video frame, and the extracted pixel points are projected to a corresponding three-dimensional coordinate system in the same manner to generate a third space projection;

[0086] Then, the overlap between the first spatial projection and the third spatial projection is calculated, so as to obtain the similarity between the first spatial projection and the third spatial projection, thereby obtaining the static similarity between the two video frames;

[0087] Then, in a similar manner, the dynamic similarity between the two video frames is calculated;

[0088] Finally, a comprehensive similarity is calculated based on the static similarity, the first preset weight corresponding to the static similarity, the dynamic similarity, and the second preset weight corresponding to the dynamic similarity, and the comprehensive similarity is used as the similarity between the first video frame and the second video frame; the schematic specific calculation formula is as follows:

[0089] A=b*B+d*D; wherein b is the first preset weight, B is the static similarity, d is the second preset weight, and D is the dynamic similarity; b>d; b+d=1;

[0090] By increasing the weight of static similarity and reducing the weight of dynamic similarity, the influence of dynamic areas on the similarity between two video frames can be weakened, which is more in line with actual monitoring scenarios and improves the accuracy of monitoring equipment fault detection.

[0091] In a preferred embodiment, the determining of the static image area and the dynamic image area of ​​the first video frame includes:

[0092] Determine the monitoring position of the monitoring device at the previous detection moment according to the first standard image corresponding to the first video frame in the stitched image, and obtain the target monitoring position;

[0093] Obtain each historical video frame collected by the monitoring device at the target monitoring position at each historical moment; generate a historical video group according to each historical video frame; wherein each historical video group includes two historical video frames that are adjacent in time sequence;

[0094] For each historical video group, each historical video frame in the historical video group is divided into a number of blocks, the second similarity of the blocks corresponding to the two historical video frames is calculated one by one, and the area corresponding to the block group whose second similarity is less than the second threshold is used as the dynamic area to be selected corresponding to the historical video group;

[0095] Taking the intersection of the to-be-selected dynamic regions corresponding to all historical video groups, to obtain the dynamic image region of the first video frame;

[0096] All image regions except the dynamic image region in the first video frame are regarded as static image regions.

[0097] First, we need to explain the stitching image:

[0098] like Figure 2 As shown, in the present invention, in the absence of any fault, the monitoring device is controlled to move in the vertical and horizontal directions and take the corresponding video frames at each monitoring position, and these video frames are spliced ​​according to the position to form Figure 2 The stitched image shown; Figure 2In the stitched image shown, assuming that video frame A is the initial position, the video frames in each row represent the video frames collected when the monitoring device moves in the horizontal dimension, and the video frames in each column represent the video frames collected when the monitoring device moves in the vertical dimension. The monitoring positions corresponding to two adjacent video frames are adjacent. The entire stitched image represents the images of all monitoring areas that can be covered by the monitoring device in a historical period without any failure.

[0099] When identifying the static image area and the dynamic image area of ​​the first video frame, the similarity between the first video frame and each video frame in the stitched image is calculated. When calculating this similarity, the existing similarity calculation method can be used. For example, the pixel values ​​of all pixels in the first video frame are extracted, and then the pixel values ​​of all pixels in a video frame to be compared in the stitched image are extracted. Then, the proportion of pixels with the same pixel values ​​in corresponding positions between the first video frame and the video frame to be compared to the total pixel values ​​is counted, and this proportion is used as the similarity between the first video frame and the video frame to be compared in the stitched image. Through the above method, the image with the highest similarity to the first video frame is selected from the stitched image to obtain the first standard image corresponding to the first video frame in the stitched image; then, according to the first standard image corresponding to the first video frame in the stitched image, the corresponding monitoring position is determined to obtain the target monitoring position. This monitoring position is the position where the monitoring device is located when collecting the first video frame.

[0100] Next, according to the historical monitoring data corresponding to the target monitoring position, each historical video frame collected by the monitoring device at the target monitoring position at each historical moment is obtained; then the historical video groups are divided so that each historical video group contains two historical video frames that are adjacent in time sequence;

[0101] For each historical video group, each historical video frame in the historical video group is divided into several blocks. When dividing, the actual size of the video frame and the preset number of divided blocks (for example, divided into 64 blocks) can be used to divide and obtain each block; then the similarity between the corresponding blocks of the two historical video frames is calculated one by one, and the similarity calculation of the times is also based on the existing method, for example, the pixel values ​​of all pixels in one block are extracted, and then the pixel values ​​of all pixels in another block are extracted, and then the proportion of pixels with the same pixel values ​​in the corresponding positions of the two blocks is counted to the total pixel points, and this proportion is used as the similarity of the two blocks, that is, the above-mentioned second similarity is obtained, and the area corresponding to the block group with the second similarity less than the second threshold is used as the dynamic area to be selected corresponding to the historical video group. Through this step, the area where the pixel changes between the previous and next two video frames in each historical video group can be screened out. Since the position of the monitoring device is the same, the reason for the pixel change is largely due to the presence of a dynamically changing scene in the area corresponding to the monitoring position.

[0102] Then, the intersection of the dynamic areas to be selected corresponding to the historical video groups is taken to obtain the dynamic image area of ​​the first video frame; by taking the intersection of the dynamic areas to be selected of the historical video groups, the influence of the picture changes caused by some accidental events can be eliminated. For example, a puppy suddenly appears in the monitoring area at a certain historical moment and then disappears quickly. In this way, the puppy will only appear in individual historical video frames. By taking the intersection of the dynamic areas to be selected of the historical video groups, the areas in the monitoring area that are in dynamic change for a long time can be accurately identified, and then the dynamic image area of ​​the first video frame is determined; finally, all image areas except the dynamic image area in the first video frame are taken as static image areas, and the division of the static image area and the dynamic image area in the first video frame is completed.

[0103] S2. If not, it is determined that the monitoring device has an image acquisition failure;

[0104] If so, the theoretical monitoring position that the monitoring device should be set at each moment in the interval period is determined according to the position adjustment instruction; according to each theoretical monitoring position, the video frame at the corresponding position is identified from the preset spliced ​​image to obtain each first standard video frame; the center point of each first standard video frame is extracted, and according to the moment corresponding to the theoretical monitoring position, the center points of each first standard video are connected in chronological order to generate the theoretical image path that the monitoring device should correspond to when moving according to the position adjustment instruction; wherein, the video frames shot by the monitoring device at all monitoring positions in the historical period are spliced ​​according to the corresponding monitoring positions to generate a spliced ​​image;

[0105] Specifically, if the monitoring device has not received a position adjustment instruction within the interval period, it means that the sudden change of the picture is caused by a fault, and it is determined that the monitoring device has an image acquisition fault;

[0106] In order to further determine the type and cause of image acquisition failure, further identification can be performed in the following ways:

[0107] In a preferred embodiment, when it is determined that the monitoring device has an image acquisition failure, it also includes: extracting the RGB value of each pixel in the second video frame; if the RGB values ​​of all pixels are all zero values, it is determined that the monitoring device has a black screen failure; if the RGB values ​​of all pixels are 255, it is determined that the monitoring device has a white screen failure; otherwise, it is determined that the monitoring device has been illegally displaced or suffered an image tampering attack.

[0108] In a preferred embodiment, after determining that a black screen failure occurs in the monitoring device, the method further includes:

[0109] According to each actual video frame actually collected by the monitoring device at each moment of the interval period, it is determined whether there is a human object that continuously approaches the monitoring device;

[0110] If so, it is determined that the black screen failure is caused by malicious human blocking; if not, it is determined that the black screen failure is caused by damage to the monitoring equipment.

[0111] If the monitoring device has received a position adjustment instruction during the interval period, it means that the sudden change of the picture is caused by the position adjustment of the monitoring device;

[0112] At this time, first determine the theoretical monitoring position that the monitoring device should be set to at each moment in the interval period according to the position adjustment instruction; specifically, take the monitoring position corresponding to the monitoring device at the previous detection moment as the initial monitoring position, and then determine the position where the monitoring device should be theoretically located at each moment in the process of adjusting the monitoring device from the initial monitoring position to the final position according to the position adjustment instruction (for example, every other frame as a moment), the position where the monitoring device should be theoretically located at each moment between the initial moment (that is, the previous detection moment) and the moment when the position adjustment instruction is received, and the position where the monitoring device should be theoretically located at each moment between the moment when the monitoring device is adjusted to the final position and the termination moment (that is, the current detection moment), so as to obtain the theoretical monitoring position that the monitoring device should be set to at each moment in the interval period;

[0113] Then, according to each theoretical monitoring position, the video frame at the corresponding position is identified from the preset spliced ​​image. Figure 3As shown, assuming that the monitoring device at the previous detection moment was at the position corresponding to the video frame A of the stitched image, it received a position adjustment instruction at the previous detection moment, and the received position adjustment instruction indicated that the monitoring device needs to move two units horizontally to the right first, and then move three units upward in the vertical direction, and the moment when the adjustment is completed coincides with the current detection moment, then the theoretical monitoring position corresponding to each moment in the interval period at this time, the corresponding first standard video frames in the stitched image are video frame A, video frame B, video frame C, video frame D, video frame E and video frame F, and then the image center points of video frame A, video frame B, video frame C, video frame D, video frame E and video frame F are connected in sequence to obtain the above-mentioned theoretical image path L1.

[0114] S3, calculating the similarity between each actual video frame actually collected by the monitoring device and each video frame in the spliced ​​image at each time point in the interval period, taking the video frame with the highest similarity in the spliced ​​image as the second standard video frame corresponding to the actual video frame, extracting the center point of each second standard video frame, connecting the center points of each second standard video frame in chronological order, and generating the actual image path corresponding to the actual movement of the monitoring device;

[0115] Specifically, after determining the theoretical image path L1, obtain each actual video frame actually collected by the monitoring device at each moment of the interval period. These actual video frames are the video frames actually collected by the monitoring device at each moment during the interval period. If the monitoring device correctly adjusts the position according to the position adjustment instruction, then the actual movement trajectory and the theoretical movement trajectory should be consistent at this time, corresponding to the spliced ​​image, that is, the theoretical image path should be consistent with the actual image path. Based on this judgment principle, for each actual video frame, the similarity between the actual video frame and each video frame in the spliced ​​image is calculated, and the video frame with the highest similarity in the spliced ​​image is used as the second standard video frame corresponding to the actual video frame. In this way, the corresponding relationship between the image captured during the actual movement of the monitoring device and the video frame in the spliced ​​image can be determined, so as to determine the various monitoring position points passed by the monitoring device during the actual movement without other positioning devices. Then connect the image center points of each second standard video frame to generate the actual image path. It should be noted that when calculating the similarity between the actual video frame and each video frame in the stitched image, the existing method is also used for calculation, schematically: the pixel values ​​of all pixels in the actual video frame are extracted, and then the pixel values ​​of all pixels in a video frame in the stitched image are extracted, and then the proportion of pixels with the same pixel values ​​at corresponding positions between the actual video frame and a video frame in the stitched image to the total pixel points is counted, and this proportion is used as the similarity between the actual video frame and a video frame in the stitched image.

[0116] Indicatively, Figure 4As shown, assuming that the monitoring device accurately adjusts its position according to the position adjustment instruction, the corresponding second standard video frames are: video frame A, video frame B, video frame C, video frame D, video frame E and video frame F, and then the image center points of video frame A, video frame B, video frame C, video frame D, video frame E and video frame F are connected in sequence to obtain the above theoretical image path L2;

[0117] Indicatively, Figure 5 As shown, assuming that the monitoring device does not correctly adjust its position according to the position adjustment instruction, it first moves horizontally to the right by two units, and then moves vertically downward by three units. At this time, the corresponding second standard video frames are: video frame A, video frame B, video frame C, video frame G, video frame H and video frame I. Then, the image center points of video frame A, video frame B, video frame C, video frame G, video frame H and video frame I are connected in order to obtain the above theoretical image path L2;

[0118] S4. Determine whether the theoretical image path and the actual image path are consistent. If they are consistent, it is determined that the monitoring device has no fault. If they are inconsistent, it is determined that the monitoring device has a movement fault.

[0119] Specifically, the stitched image with the theoretical image path (such as Figure 3 ) and the stitched image with the actual image path (such as Figure 4 ) After the theoretical image path and the actual image path are overlapped, the comparison is made to see whether they are completely overlapped. If so, it is determined that the theoretical image path and the actual image path are consistent, which means that the monitoring device moves accurately according to the position adjustment instruction, and the second video frame collected at the current detection time is also correct, and the monitoring device has no fault. Otherwise, it is determined that the theoretical image path and the actual image path are inconsistent, and it is determined that the monitoring device has a movement fault.

[0120] In a preferred embodiment, when it is determined that the monitoring device has a movement failure, the path point at which the theoretical image path and the actual image path begin to differ is determined according to the overlap between the theoretical image path and the actual image path, and the difference path point is obtained;

[0121] The time and position when the monitoring device has a movement failure are determined according to the time and position corresponding to the video frame where the difference path point is located.

[0122] Specifically, assume that the actual image path generated is Figure 5 As shown, Figure 5 and Figure 3 After comparison, the difference path points can be determined as Figure 6 The center point of the video frame C;

[0123] According to the time and position corresponding to the video frame C, the specific time and position when the monitoring device has a mobile failure can be determined.

[0124] In a preferred embodiment, after determining that the monitoring device has a movement failure, the method further includes:

[0125] Determine the position of the first standard video frame corresponding to the previous detection moment and the current detection moment in the spliced ​​image, and obtain a first position and a second position respectively;

[0126] According to the first position and the second position, determining in the stitched image an image path that avoids the difference path point and is the shortest when traveling from the first position to the second position, to obtain a target image path;

[0127] The monitoring device is controlled to move in sequence according to the monitoring positions corresponding to the target image path to complete the position correction.

[0128] In this embodiment, position correction is required after a movement failure occurs. At this time, the position of the first standard video frame corresponding to the previous detection moment and the current detection moment in the spliced ​​image can be determined first, and the initial position (i.e., the first position) and the final position (i.e., the second position) when adjusting according to the position adjustment instruction are obtained;

[0129] Then, through enumeration, all image paths from the first position to the second position are traversed on the spliced ​​image, and then the image path that does not contain the difference path point and has the shortest path is selected to obtain the target image path. Finally, the monitoring device is controlled to move in sequence according to the monitoring positions corresponding to the target image path to complete the position correction. Using this position correction method, the difference path point can be avoided. Since the theoretical image path and the actual image path begin to differ at the difference path point, this means that there may be obstacles at the monitoring position of the video frame corresponding to the difference path point, resulting in the monitoring position being unable to continue to be adjusted here. At this time, when correcting the position, the target image path that avoids the difference path point is selected for position correction, which can improve the success rate of position correction.

[0130] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0131] Please refer to Figure 7 , a schematic diagram of the structure of a security video surveillance fault detection device provided by an embodiment of the present invention, the device comprises: an instruction determination module, a first fault identification module, a theoretical image path generation module, an actual image path generation module and a second fault identification module;

[0132] The instruction determination module is used to obtain a first video frame collected by the monitoring device at a previous detection moment and a second video frame collected at a current detection moment; if it is determined that the first similarity between the first video frame and the second video frame is less than a first threshold, detect whether the monitoring device receives a position adjustment instruction during an interval between the previous detection moment and the current detection moment;

[0133] The first fault identification module is used to determine that an image acquisition fault occurs in the monitoring device when it is determined that the monitoring device has not received the position adjustment instruction;

[0134] The theoretical image path generation module is used to determine the theoretical monitoring position that the monitoring device should be set at each moment in the interval period according to the position adjustment instruction when it is determined that the monitoring device has received the position adjustment instruction; identify the video frame at the corresponding position from the preset spliced ​​image according to each theoretical monitoring position, obtain each first standard video frame, extract the center point of each first standard video frame, and connect the center points of each first standard video in chronological order according to the moment corresponding to the theoretical monitoring position, so as to generate the theoretical image path that the monitoring device should correspond to when moving according to the position adjustment instruction; wherein, the video frames shot by the monitoring device at all monitoring positions in the historical period are spliced ​​according to the corresponding monitoring positions to generate a spliced ​​image;

[0135] The actual image path generation module is used to calculate the similarity between each actual video frame actually collected by the monitoring device and each video frame in the spliced ​​image at each time of the interval period, use the video frame with the highest similarity in the spliced ​​image as the second standard video frame corresponding to the actual video frame, extract the center point of each second standard video frame, connect the center points of each second standard video frame in chronological order, and generate the actual image path corresponding to the actual movement of the monitoring device;

[0136] The second fault identification module is used to determine whether the theoretical image path and the actual image path are consistent. If they are consistent, it is determined that the monitoring device has no fault. If they are inconsistent, it is determined that the monitoring device has a movement fault.

[0137] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0138] Those skilled in the art can clearly understand that for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0139] Accordingly, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the security video surveillance fault detection method described in the above-mentioned embodiment of the invention when executing the computer program.

[0140] The electronic device may be a computing device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The device may include, but is not limited to, a processor and a memory.

[0141] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the device, and various interfaces and lines are used to connect various parts of the entire device.

[0142] Accordingly, an embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the security video surveillance fault detection method described in the above-mentioned embodiment of the invention.

[0143] The memory can be used to store the computer program, and the processor realizes various functions of the device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0144] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each method embodiment described above can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0145] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A security video monitoring fault detection method, characterized in that: include: Acquire a first video frame captured by the monitoring device at a previous detection moment and a second video frame captured at a current detection moment; if it is determined that a first similarity between the first video frame and the second video frame is less than a first threshold, detect whether the monitoring device receives a position adjustment instruction during an interval between the previous detection moment and the current detection moment; If not, it is determined that the monitoring device has an image acquisition failure; If so, the theoretical monitoring position that the monitoring device should be set at each moment in the interval period is determined according to the position adjustment instruction; according to each theoretical monitoring position, the video frame at the corresponding position is identified from the preset spliced ​​image to obtain each first standard video frame; the center point of each first standard video frame is extracted, and according to the moment corresponding to the theoretical monitoring position, the center points of each first standard video are connected in chronological order to generate the theoretical image path that the monitoring device should correspond to when moving according to the position adjustment instruction; wherein, the video frames shot by the monitoring device at all monitoring positions in the historical period are spliced ​​according to the corresponding monitoring positions to generate a spliced ​​image; Calculate the similarity between each actual video frame actually collected by the monitoring device and each video frame in the spliced ​​image at each moment of the interval period, use the video frame with the highest similarity in the spliced ​​image as the second standard video frame corresponding to the actual video frame, extract the center point of each second standard video frame, connect the center points of each second standard video frame in chronological order, and generate the actual image path corresponding to the actual movement of the monitoring device; It is determined whether the theoretical image path and the actual image path are consistent. If they are consistent, it is determined that the monitoring device has no fault. If they are inconsistent, it is determined that the monitoring device has a movement fault.

2. The security video surveillance fault detection method according to claim 1, characterized in that: The first similarity between the first video frame and the second video frame is calculated in the following manner: Determine a static image area and a dynamic image area of ​​a first video frame; A three-dimensional coordinate system is constructed with the row coordinates of the pixel point in the image as the X-axis, the column coordinates of the pixel point in the image as the Y-axis, and the pixel value of the pixel point as the Y-axis; Projecting pixel points in the static image area of ​​the first video frame into a three-dimensional coordinate system to generate a first spatial projection; Projecting the pixel points in the dynamic image area of ​​the first video frame into a three-dimensional coordinate system to generate a second spatial projection; Projecting the pixel points at the corresponding position in the second video frame into a three-dimensional coordinate system according to the static image area of ​​the first video frame to generate a third space projection; Projecting the pixel points at the corresponding positions in the second video frame into a three-dimensional coordinate system according to the dynamic image area of ​​the first video frame to generate a fourth spatial projection; Calculating the similarity between the first spatial projection and the third spatial projection to obtain a static similarity; Calculating the similarity between the second spatial projection and the fourth spatial projection to obtain dynamic similarity; The comprehensive similarity is calculated according to the static similarity, the first preset weight corresponding to the static similarity, the dynamic similarity and the second preset weight corresponding to the dynamic similarity; wherein the first preset weight is greater than the second preset weight; The comprehensive similarity is used as a first similarity between the first video frame and the second video frame.

3. The security video surveillance fault detection method according to claim 2, characterized in that: The determining of the static image area and the dynamic image area of ​​the first video frame includes: Determine the monitoring position of the monitoring device at the previous detection moment according to the first standard image corresponding to the first video frame in the stitched image, and obtain the target monitoring position; Obtain each historical video frame collected by the monitoring device at the target monitoring position at each historical moment; generate a historical video group according to each historical video frame; wherein each historical video group includes two historical video frames that are adjacent in time sequence; For each historical video group, each historical video frame in the historical video group is divided into a number of blocks, the second similarity of the blocks corresponding to the two historical video frames is calculated one by one, and the area corresponding to the block group whose second similarity is less than the second threshold is used as the dynamic area to be selected corresponding to the historical video group; Taking the intersection of the to-be-selected dynamic regions corresponding to all historical video groups, to obtain the dynamic image region of the first video frame; All image regions except the dynamic image region in the first video frame are regarded as static image regions.

4. The security video surveillance fault detection method according to claim 3, characterized in that: Also includes: When it is determined that the monitoring device has a movement failure, the path point at which the theoretical image path and the actual image path begin to differ is determined according to the overlap between the theoretical image path and the actual image path, to obtain a difference path point; The time and position when the monitoring device has a movement failure are determined according to the time and position corresponding to the video frame where the difference path point is located.

5. The security video surveillance fault detection method according to claim 4, characterized in that: After determining that the monitoring device has a movement failure, the method further includes: Determine the position of the first standard video frame corresponding to the previous detection moment and the current detection moment in the spliced ​​image, and obtain a first position and a second position respectively; According to the first position and the second position, determining in the stitched image an image path that avoids the difference path point and is the shortest when traveling from the first position to the second position, to obtain a target image path; The monitoring device is controlled to move in sequence according to the monitoring positions corresponding to the target image path to complete the position correction.

6. The security video surveillance fault detection method according to claim 5, characterized in that: In the case where it is determined that the monitoring device has an image acquisition failure, the method further includes: Extracting the RGB value of each pixel in the second video frame; If the RGB values ​​of all pixels are all zero, it is determined that the monitoring device has a black screen failure; If the RGB values ​​of all pixels are 255, it is determined that the monitoring device has a white screen failure; Otherwise, it is determined that the monitoring device has been illegally moved or suffered from an image tampering attack.

7. The security video surveillance fault detection method according to claim 6, characterized in that: After confirming that the monitoring device has a black screen failure, it also includes: According to each actual video frame actually collected by the monitoring device at each moment of the interval period, it is determined whether there is a human object that continuously approaches the monitoring device; If so, it is determined that the black screen failure is caused by malicious human blocking; if not, it is determined that the black screen failure is caused by damage to the monitoring equipment.

8. A security video monitoring fault detection device, characterized in that: include: An instruction determination module, a first fault identification module, a theoretical image path generation module, an actual image path generation module, and a second fault identification module; The instruction determination module is used to obtain a first video frame collected by the monitoring device at a previous detection moment and a second video frame collected at a current detection moment; if it is determined that the first similarity between the first video frame and the second video frame is less than a first threshold, detect whether the monitoring device receives a position adjustment instruction during an interval between the previous detection moment and the current detection moment; The first fault identification module is used to determine that an image acquisition fault occurs in the monitoring device when it is determined that the monitoring device has not received the position adjustment instruction; The theoretical image path generation module is used to determine the theoretical monitoring position that the monitoring device should be set at each moment in the interval period according to the position adjustment instruction when it is determined that the monitoring device has received the position adjustment instruction; identify the video frame at the corresponding position from the preset spliced ​​image according to each theoretical monitoring position, obtain each first standard video frame, extract the center point of each first standard video frame, and connect the center points of each first standard video in chronological order according to the moment corresponding to the theoretical monitoring position, so as to generate the theoretical image path that the monitoring device should correspond to when moving according to the position adjustment instruction; wherein, the video frames shot by the monitoring device at all monitoring positions in the historical period are spliced ​​according to the corresponding monitoring positions to generate a spliced ​​image; The actual image path generation module is used to calculate the similarity between each actual video frame actually collected by the monitoring device and each video frame in the spliced ​​image at each time of the interval period, use the video frame with the highest similarity in the spliced ​​image as the second standard video frame corresponding to the actual video frame, extract the center point of each second standard video frame, connect the center points of each second standard video frame in chronological order, and generate the actual image path corresponding to the actual movement of the monitoring device; The second fault identification module is used to determine whether the theoretical image path and the actual image path are consistent. If they are consistent, it is determined that the monitoring device has no fault. If they are inconsistent, it is determined that the monitoring device has a movement fault.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the security video surveillance fault detection method according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the security video surveillance fault detection method according to any one of claims 1 to 7.

Images