Intelligent video monitoring early warning method and system, electronic equipment and storage medium

Abstract

The invention discloses an intelligent video monitoring early warning method and system, electronic equipment and a storage medium, and relates to the technical field of video surveillance, and the method comprises the steps: shooting a preset region through a preset camera, and carrying out the target recognition of images collected at two adjacent moments, and obtaining a detection frame of each recognized target; obtaining the position change of the detection frame of any identified preset target, and determining whether to send out alarm information or not according to the position change; wherein a target jointly contained by the images collected at the two adjacent moments is used as a preset target. According to the invention, based on the position variation, static non-risk targets can be automatically screened out, the number of invalid alarms is effectively reduced, the alarm accuracy is improved, the workload of personnel verification and disposal is effectively reduced, and the working efficiency is improved.

Description

Background Art

[0002] At present, some enterprises have installed more than a thousand video surveillance facilities along the pipeline. The existing intelligent video surveillance system captures the real-time images of all front-end cameras every 10 minutes, and analyzes the risk information such as excavators, personnel staying, suspicious vehicles, etc. in the images through algorithms. If there is risk information, the intelligent video surveillance system will push an alarm. After receiving the alarm, the on-duty personnel will first determine whether there is a risk through the on-site images. If there is no risk, the alarm will be directly confirmed through the system. If there is a risk or it cannot be verified through video surveillance, it is necessary to notify the line management personnel along the line to verify on-site to ensure that the risk event is handled in a timely manner and to ensure the safety of the pipeline.

[0003] However, the existing intelligent video surveillance system will repeatedly push alarm information for static risk-free targets such as parked vehicles, stationary machinery, oil pumps, valve room pipelines, leaves at night, etc., resulting in a high proportion of invalid alarms. Specifically, an average of more than 20,000 alarm messages are generated every day, of which about 90% are repeated risk-free invalid alarm messages. The workload of personnel verification and disposal is large, and the work efficiency is low. Summary of the invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies of the prior art and specifically provide an intelligent video monitoring early warning method, system, electronic device and storage medium, as follows:

[0005] 1) In the first aspect, the present invention provides an intelligent video surveillance early warning method, and the specific technical solution is as follows:

[0006] The preset area is photographed by a preset camera, and target recognition is performed on the images collected at two adjacent moments to obtain a detection frame of each identified target;

[0007] Obtain and determine whether to issue an alarm message based on the position change of the detection frame of any preset target identified;

[0008] The target commonly contained in the images collected at two adjacent moments is taken as the preset target.

[0009] The beneficial effects of an intelligent video monitoring early warning method provided by the present invention are as follows:

[0010] Based on position changes, it can automatically screen out static risk-free targets, effectively reduce the number of invalid alarms, improve the accuracy of alarms, effectively reduce the workload of personnel verification and disposal, and improve work efficiency.

[0011] On the basis of the above scheme, the intelligent video monitoring and early warning method of the present invention can also be improved as follows.

[0012] Furthermore, it also includes:

[0013] Traverse and obtain, and determine whether to issue an alarm message based on the position change of the detection frame of each preset target.

[0014] Furthermore, it also includes:

[0015] It is determined whether each target contained in the image collected at the latter moment of two adjacent moments is all the preset targets, and whether to issue an alarm message is determined according to the determination result.

[0016] The beneficial effects of adopting the above further technical solution are as follows: the accuracy of the alarm can be further improved.

[0017] Furthermore, the preset area is: a local area along the pipeline.

[0018] 2) In the second aspect, the present invention also provides an intelligent video monitoring early warning system, and the specific technical solution is as follows:

[0019] Including image recognition module and early warning module;

[0020] The image recognition module is used to: shoot a preset area through a preset camera, and perform target recognition on the images collected at two adjacent moments to obtain a detection frame of each recognized target;

[0021] The early warning module is used to: obtain and determine whether to issue an alarm message based on the position change of the detection frame of any preset target identified;

[0022] The target commonly contained in the images collected at two adjacent moments is taken as the preset target.

[0023] On the basis of the above scheme, the intelligent video monitoring and early warning system of the present invention can also be improved as follows.

[0024] Furthermore, it also includes a traversal calling module, which is used to:

[0025] The early warning module is called repeatedly to traverse and obtain the position changes of the detection frame of each preset target, and determine whether to issue an alarm message.

[0026] Furthermore, the early warning module is also used to:

[0027] It is determined whether each target contained in the image collected at the latter moment of two adjacent moments is all the preset targets, and whether to issue an alarm message is determined according to the determination result.

[0028] Furthermore, the preset area is: a local area along the pipeline.

[0029] 3) In a third aspect, the present invention further provides an electronic device, comprising a processor, the processor being coupled to a memory, the memory storing at least one computer program, and the at least one computer program being loaded and executed by the processor so that the electronic device can implement any one of the above-mentioned intelligent video surveillance and early warning methods.

[0030] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored, and at least one computer program is loaded and executed by a processor so that a computer implements any one of the above-mentioned intelligent video surveillance and early warning methods.

[0031] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0033] Figure 1 A schematic diagram of a process of an intelligent video monitoring and early warning method according to an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of the structure of an intelligent video monitoring and early warning system according to an embodiment of the present invention;

[0035] Figure 3 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0037] like Figure 1 As shown, an intelligent video monitoring early warning method according to an embodiment of the present invention comprises the following steps:

[0038] S1, photographing a preset area through a preset camera, and performing target recognition on the images collected at two adjacent moments to obtain a detection frame of each recognized target;

[0039] Among them, the preset area is: a local area along the pipeline. Multiple cameras can be deployed along the pipeline. The preset camera is any camera. The expected area photographed by the preset camera is within the shooting range of the preset camera.

[0040] For ease of expression, two adjacent moments are respectively recorded as: the first moment and the second moment, the first moment is the previous moment of the two adjacent moments, and the second moment is the latter moment of the two adjacent moments. The image taken at the first moment is recorded as the first image, and the image taken at the second moment is recorded as the second image. The sizes of the first image and the second image are the same.

[0041] The detection frame may be a rectangular detection frame, or a detection frame of other shapes may be set according to actual conditions.

[0042] S2. Obtain and determine whether to issue an alarm message based on the position change of the detection frame of any preset target identified, specifically:

[0043] 1) The overlapping area is used to characterize the magnitude of the position change, and the overlapping area of ​​the detection frames of the same preset target in the first image and the second image is calculated. When the proportion of the overlapping area is greater than the preset proportion threshold, it means that the position change of the preset target between two adjacent moments is small, and no alarm information associated with the preset target is issued; when the proportion of the overlapping area is not greater than the preset proportion threshold, it means that the position change of the preset target between two adjacent moments is large, and an alarm information associated with the preset target is issued.

[0044] The overlapping area may be represented by the number of pixel points, or a two-dimensional coordinate system may be established on the captured image, and the overlapping area may be calculated in the two-dimensional coordinate system.

[0045] For example, when the detection frames are all rectangular detection frames, the rectangular detection frames of the same preset target in the first image and the second image are respectively recorded as the first rectangular detection frame and the second rectangular detection frame. The first rectangular detection frame and the second rectangular detection frame can be placed in the same image according to the relative positions (the position of the first rectangular detection frame relative to the first image, and the position of the second rectangular detection frame relative to the second image). A two-dimensional coordinate system is established in the image, the Y axis of the two-dimensional coordinate system is along the column direction of the pixels, and the X axis of the two-dimensional coordinate system is along the row direction of the pixels. The two opposite sides of the identified rectangular detection frames (the first rectangular detection frame and the second rectangular detection frame) are parallel to the X axis of the two-dimensional coordinate system, and the other two opposite sides are parallel to the Y axis of the two-dimensional coordinate system. Then:

[0046] S20, determining overlapping areas:

[0047] The coordinates of the overlapping area depend on the relative positions of the first rectangular detection frame and the second rectangular detection frame. If the first rectangular detection frame and the second rectangular detection frame overlap, the coordinates of the overlapping area will be the intersection of the coordinates of the first rectangular detection frame and the second rectangular detection frame. Let the coordinates of the lower left corner of the overlapping area be (X overlap_min ,Y overlap_min ), the coordinate of the upper right corner is (X overlap_max ,Yoverlap_max ).

[0048] S21. Calculate the overlapping area:

[0049] By the formula Aoverlap = (X max -X min )×(Y max -Y min ) Calculate the overlap area Aoverlap, X min and Y min is the minimum coordinate of the overlapping area, X max and Y max are the maximum coordinates of the overlapping area.

[0050] S22. Use Percentage = (Aoverlap / Aoriginal) × 100% to calculate the percentage of the overlapping area to the original area Aoriginal (i.e., the area of ​​the first rectangular detection box). If the overlapping area ratio is not less than the preset ratio threshold, no alarm message is issued. If the overlapping area ratio is greater than the preset ratio threshold, an alarm message is issued.

[0051] 2) When the detection frame is a rectangular detection frame, the change in the positions of the four corner points of the rectangular detection frame is used to represent the magnitude of the position change, and the displacement of the four corner points of the detection frame of the same preset target in the first image and the second image is calculated. When the displacement of a corner point position is greater than a preset displacement threshold, it indicates that the position change is large, and an alarm message associated with the preset target is issued. When the displacement of all corner point positions is not greater than the preset displacement threshold, it indicates that the position change is small, and no alarm message associated with the preset target is issued.

[0052] The displacement may be represented by the number of intervals between pixels in the images (the first image and the second image), or a two-dimensional coordinate system may be established on the captured images, and the displacement may be calculated in the two-dimensional coordinate system.

[0053] The target commonly contained in the images collected at two adjacent moments is taken as the preset target.

[0054] Optionally, in the above technical solution, it also includes:

[0055] S3, traverse and obtain and determine whether to issue an alarm message according to the position change of the detection frame of each preset target, specifically:

[0056] Traverse to obtain the position change of the detection frame of each preset target, and obtain the first judgment result corresponding to each preset target. When all the first judgment results are negative, do not issue an alarm message associated with each preset target. When all the first judgment results are not negative, issue an alarm message associated with the target corresponding to each negative first judgment result.

[0057] Optionally, in the above technical solution, it also includes:

[0058] S4, judging whether all the targets contained in the image collected at the later moment of two adjacent moments are all preset targets, and determining whether to issue an alarm message according to the judgment result, specifically:

[0059] Determine whether each target contained in the second image is all preset targets, and obtain a judgment result. When the judgment result is no, issue an alarm message associated with each new target, wherein the new target is: a target other than all preset targets in the second image (the image captured at the later moment of two adjacent moments).

[0060] The present invention is further described by the following examples, which specifically include:

[0061] S10. Upgrading the image recognition algorithm in the existing intelligent video surveillance system, specifically:

[0062] The upgraded image recognition algorithm identifies and marks the position of each automatically identified target (also called risk target) in the image, specifically marking the horizontal coordinate X and vertical coordinate Y of the four corner positions of the rectangular detection frame of each target. The horizontal coordinate X is the number of columns of the pixel point at the corner position in the image, and the vertical coordinate Y is the number of rows of the pixel point at the corner position in the image. These coordinates are pushed together with the corresponding alarm information (risk alarm information) to the alarm screening program in the existing intelligent video surveillance system (a secondary recognition program used to determine whether the alarm information needs to be pushed to the client). At the same time, the information pushed also includes the target type, target quantity, etc. When multiple targets appear in the same image, the targets are numbered logically from left to right and from top to bottom.

[0063] S11. Optimize the alarm push logic in the alarm screening program, and upgrade the previous method of identifying each photo in isolation to the method of identifying the association of the previous and next photos. Specifically:

[0064] The upgraded image recognition algorithm is used to identify the target in the image taken by the same camera at the same preset position, and the target recognition result is compared with the target recognition result of the previous image. If the type of target, the number of targets, and the coordinate position of the target in the photo are the same, and the position changes slightly, it is considered to be the same as the previous alarm, and the alarm information is no longer pushed, thereby reducing invalid alarms. When the position of any of the same targets (preset targets) changes greatly, it is considered a dynamic target and an alarm is pushed to improve work efficiency. The calculation steps are as follows:

[0065] When the detection frames are all rectangular detection frames, the rectangular detection frames of the same preset target in the first image and the second image are respectively recorded as the first rectangular detection frame and the second rectangular detection frame. The first rectangular detection frame and the second rectangular detection frame can be placed in the same image according to the relative positions (the position of the first rectangular detection frame relative to the first image, and the position of the second rectangular detection frame relative to the second image). A two-dimensional coordinate system is established in the image, the Y axis of the two-dimensional coordinate system is along the column direction of the pixels, and the X axis of the two-dimensional coordinate system is along the row direction of the pixels. The two opposite sides of the identified rectangular detection frames (the first rectangular detection frame and the second rectangular detection frame) are parallel to the X axis of the two-dimensional coordinate system, and the other two opposite sides are parallel to the Y axis of the two-dimensional coordinate system. Then:

[0066] 1. Determine the overlapping area:

[0067] The coordinates of the overlapping area depend on the relative positions of the first rectangular detection frame and the second rectangular detection frame. If the first rectangular detection frame and the second rectangular detection frame overlap, the coordinates of the overlapping area will be the intersection of the coordinates of the first rectangular detection frame and the second rectangular detection frame. Let the coordinates of the lower left corner of the overlapping area be (X overlap_min ,Y overlap_min ), the coordinate of the upper right corner is (X overlap_max ,Y overlap_max ).

[0068] 2. Calculate the overlapping area:

[0069] By the formula Aoverlap = (X max -X min )×(Y max -Y min ) Calculate the overlap area Aoverlap, X min and Y min is the minimum coordinate of the overlapping area, X max and Y max are the maximum coordinates of the overlapping area.

[0070] 3. Use Percentage = (Aoverlap / Aoriginal) × 100% to calculate the percentage of the overlapping area to the original area Aoriginal (i.e., the area of ​​the first rectangular detection frame): If the overlapping area ratio is not less than 90%, it is determined to be the same alarm. In this case, it will not be pushed to the user repeatedly.

[0071] Optionally, in the above technical solution, the coordinates of the lower left corner point of the first rectangular detection frame are (X1, Y1), the coordinates of the upper right corner point of the first rectangular detection frame are (X2, Y2), the coordinates of the lower left corner point of the second rectangular detection frame are (X3, Y3), and the coordinates of the upper right corner point of the first rectangular detection frame are (X4, Y4); the relative position relationship between the first rectangular detection frame and the second rectangular detection frame is determined based on the difference between X1 and X3, the difference between Y1 and Y3, the difference between X2 and X4, and the difference between Y2 and Y4, and the relative position relationship reflects the moving direction of the target.

[0072] In the embodiment of the present invention, by upgrading the image recognition algorithm and optimizing the alarm screening program, static risk-free targets such as parked excavators, valve room pipelines, infusions, etc. can be automatically screened out, effectively reducing the number of invalid alarms, improving the accuracy of alarms, effectively reducing the workload of personnel verification and disposal, and improving work efficiency.

[0073] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0074] like Figure 2 As shown, an intelligent video monitoring and early warning system 200 according to an embodiment of the present invention includes an image recognition module 201 and an early warning module 202;

[0075] The image recognition module 201 is used to: shoot a preset area through a preset camera, and perform target recognition on the images collected at two adjacent moments to obtain a detection frame of each recognized target;

[0076] The early warning module 203 is used to: obtain and determine whether to issue an alarm message according to the position change of the detection frame of any preset target identified;

[0077] The target commonly contained in the images collected at two adjacent moments is taken as the preset target.

[0078] Optionally, in the above technical solution, a traversal calling module is further included, and the traversal calling module is used to:

[0079] The early warning module 202 is called repeatedly to traverse and obtain and determine whether to issue an alarm message according to the position change of the detection frame of each preset target.

[0080] Optionally, in the above technical solution, the early warning module 202 is further used for:

[0081] It is determined whether each target contained in the image collected at the latter moment of two adjacent moments is all the preset targets, and whether to issue an alarm message is determined according to the determination result.

[0082] Optionally, in the above technical solution, the preset area is: a local area along the pipeline.

[0083] It should be noted that the beneficial effects of the intelligent video surveillance warning system 200 provided in the above embodiment are the same as the beneficial effects of the intelligent video surveillance warning method described above, which will not be described in detail here. In addition, when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be described in detail here.

[0084] like Figure 3 As shown, an electronic device 300 of an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above-mentioned intelligent video monitoring early warning methods, specifically:

[0085] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements any of the intelligent video monitoring and early warning methods provided in the above embodiments. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The electronic device 300 may also include other components for realizing the functions of the device, which will not be described in detail here. The electronic device may specifically be a computer, a camera with machine vision recognition function, and a drone, etc.

[0086] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above-mentioned intelligent video surveillance early warning methods.

[0087] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0088] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes any of the above-mentioned intelligent video surveillance early warning methods.

[0089] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0090] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0091] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0092] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. An intelligent video surveillance early warning method, characterized in that: include: The preset area is photographed by a preset camera, and target recognition is performed on the images collected at two adjacent moments to obtain a detection frame of each identified target; Obtain and determine whether to issue an alarm message based on the position change of the detection frame of any preset target identified; The target commonly contained in the images collected at two adjacent moments is taken as a preset target.

2. The intelligent video monitoring early warning method according to claim 1 is characterized in that: Also includes: Traverse and obtain, and determine whether to issue an alarm message based on the position change of the detection frame of each preset target.

3. An intelligent video surveillance early warning method according to claim 1 or 2, characterized in that: Also includes: It is determined whether each target contained in the image collected at the latter moment of two adjacent moments is all the preset targets, and whether to issue an alarm message is determined according to the determination result.

4. An intelligent video monitoring early warning method according to claim 1 or 2, characterized in that: The preset area is: a local area along the pipeline.

5. An intelligent video surveillance early warning system, characterized in that: Including image recognition module and early warning module; The image recognition module is used to: shoot a preset area through a preset camera, and perform target recognition on images collected at two adjacent moments to obtain a detection frame of each recognized target; The early warning module is used to: obtain and determine whether to issue an alarm message according to the position change of the detection frame of any preset target identified; The target commonly contained in the images collected at two adjacent moments is taken as a preset target.

6. The intelligent video surveillance early warning system according to claim 5, characterized in that: It also includes a traversal calling module, which is used to: The early warning module is called repeatedly to traverse and obtain and determine whether to issue an alarm message according to the position change of the detection frame of each preset target.

7. An intelligent video monitoring and early warning system according to claim 5 or 6, characterized in that: The early warning module is also used for: It is determined whether each target contained in the image collected at the latter moment of two adjacent moments is all the preset targets, and whether to issue an alarm message is determined according to the determination result.

8. An intelligent video monitoring and early warning system according to claim 5 or 6, characterized in that: The preset area is: a local area along the pipeline.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements an intelligent video surveillance early warning method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: At least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by the processor so that the computer implements the intelligent video surveillance early warning method as described in any one of claims 1 to 4.

Images