Image capturing method, image display method, device and storage medium
By identifying and tracking target objects in video surveillance frames and delaying the capture of panoramic images that meet the moment of the capture rules, the problems of inaccurate extraction of target objects and waste of storage space in video surveillance are solved, and efficient image storage and processing are achieved .
Patent Information
- Application Number
- CN202110831445.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-22
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-07-22
AI Technical Summary
In video surveillance, it is difficult for the prior art to accurately capture the target box of the target object in a panoramic picture, and due to hardware resources and network bandwidth limitations, the image is lost and the storage space consumed is excessive.
By identifying the target object that triggers the alarm rule in the video surveillance frame and tracking its position in multiple consecutive frames, delaying the grabbing of the panoramic picture that meets the grabbing rule moment, only the last frame containing the target object is captured, and send Perform target box extraction for the backend.
Ensures that the target box of the target object is accurately extracted in the panoramic picture, reducing storage space requirements and reducing the storage pressure of the device.
Smart Images

Figure CN113591651B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method for capturing an image, an image display method, a device, and a storage medium. Background Art
[0002] In video surveillance, the camera needs to detect various target objects, such as pedestrians, motor vehicles and non-motor vehicles. When it is detected that the target object triggers certain rules, such as line-tripping rules, area intrusion rules, and item abandoned rules, it is necessary to capture the target frame including the target object and the corresponding panoramic picture in order to preserve the corresponding evidence.
[0003] In the related art, two schemes are often used to capture images. The first scheme is to randomly capture a panoramic picture at the current moment when an event occurs, that is, when a target object triggers a rule, but the disadvantage of this scheme is that the specific target object that triggers the rule cannot be displayed on the panoramic picture. The second scheme is to accurately capture a panoramic picture corresponding to the moment of the event and the corresponding target frame including the target object when an event occurs. The disadvantage of this scheme is that when there are many events, the picture will be lost due to the limitations of various hardware resources and network bandwidth, and saving the picture will also consume a lot of storage space. Summary of the invention
[0004] To solve the existing technical problems, the embodiments of the present application provide a method for capturing an image, an image display method, an apparatus and a storage medium, which can ensure that a target frame including a target object is accurately captured in the entire image, and reduce the storage space required for storing the image.
[0005] To achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:
[0006] In a first aspect, an embodiment of the present application provides a method for capturing an image, the method comprising:
[0007] If there is a target object that triggers the alarm rule in the video surveillance frame, the target object is identified in a plurality of subsequent consecutive video surveillance frames;
[0008] After the capture rule is met, a panoramic picture of the video surveillance frame that meets the capture rule at a corresponding time is captured; wherein the video surveillance frame that meets the capture rule at a corresponding time is the last video surveillance frame among the plurality of consecutive video surveillance frames; and the plurality of consecutive video surveillance frames all include the target object;
[0009] The recognition result of the panoramic picture and the video surveillance frame at the time corresponding to the capture rule is sent to the back end, so that the back end extracts the target frame including the target object in the panoramic picture according to the recognition result.
[0010] The method for capturing images provided by the embodiment of the present application, if there is a target object that triggers an alarm rule in a video surveillance frame, the target object is identified in a plurality of subsequent consecutive video surveillance frames, and after the capture rule is satisfied, a panoramic picture of the video surveillance frame that satisfies the corresponding time of the capture rule is captured, and the recognition result of the panoramic picture and the video surveillance frame at the corresponding time of the capture rule is sent to the back end, so that the back end extracts a target frame including the target object in the panoramic picture according to the recognition result. Since the panoramic picture corresponding to the video surveillance frame at the current time is not captured immediately when the alarm rule is triggered, but the panoramic picture of the surveillance video frame that satisfies the corresponding time of the capture rule is captured, and after the target object triggers the alarm rule, the target object is tracked in real time, so that only one panoramic picture needs to be captured, and it can be ensured that the back end can accurately extract the target frame including the target object when extracting the target frame.
[0011] In an optional embodiment, if there is a target object that triggers an alarm rule in a video surveillance frame, identifying the target object in a plurality of subsequent consecutive video surveillance frames includes:
[0012] If there is a target object that triggers the alarm rule in the video surveillance frame, the target identifier and position coordinates corresponding to the target object are determined respectively; the position coordinates corresponding to the target object are determined according to the first position of the target object in the video surveillance frame;
[0013] According to the target identifier corresponding to the target object, the target object is identified in a plurality of subsequent consecutive video surveillance frames, and a second position of the target object in the plurality of consecutive video surveillance frames is determined respectively;
[0014] Each time a second position is determined, the position coordinates are updated according to the determined second position.
[0015] In this embodiment, if there is a target object that triggers an alarm rule in a video surveillance frame, the target identifier and position coordinates corresponding to the target object are determined respectively, wherein the position coordinates corresponding to the target object are determined according to the first position of the target object in the video surveillance frame, and according to the target identifier corresponding to the target object, the target object is identified in a plurality of subsequent consecutive video surveillance frames, and the second position of the target object in a plurality of consecutive video surveillance frames is determined respectively, and each time a second position is determined, the position coordinates are updated according to the determined second position. Since the position of the target object in the video surveillance frame can be tracked in real time, the accuracy of updating the position coordinates corresponding to the target object can be ensured.
[0016] In an optional embodiment, identifying the target object in a plurality of subsequent consecutive video surveillance frames according to the target identifier corresponding to the target object, and respectively determining a second position of the target object in the plurality of consecutive video surveillance frames, includes:
[0017] For each of the plurality of consecutive video surveillance frames, perform the following operations respectively:
[0018] Identify the video surveillance frame to determine each object in the video surveillance frame;
[0019] According to the target identifier corresponding to the target object, the target object is matched with each of the objects to determine a second position of the target object in the video surveillance frame.
[0020] In this embodiment, for each of the multiple consecutive video surveillance frames, the following operations can be performed respectively: the video surveillance frame is identified, each object in the video surveillance frame is determined, and the target object is matched with each object according to the target identifier corresponding to the target object, and the second position of the target object in the video surveillance frame is determined. Thus, the position of the target object in the video surveillance frame can be tracked in real time.
[0021] In an optional embodiment, sending the recognition result of the video surveillance frame at the time corresponding to the capture rule to the back end includes:
[0022] The target identifier and position coordinates corresponding to the target object are sent to the back end; the position coordinates corresponding to the target object are determined according to the second position of the target object in the video surveillance frame at the time corresponding to the capture rule.
[0023] In this embodiment, the target identifier and position coordinates corresponding to the target object can be sent to the back end, wherein the position coordinates corresponding to the target object are determined according to the second position of the target object in the video surveillance frame at the time corresponding to the capture rule. Since the target identifier and position coordinates corresponding to the target object can be sent to the back end, the back end can accurately extract the target frame including the target object in the panoramic picture according to the target identifier and position coordinates corresponding to the target object.
[0024] In an optional embodiment, the capturing of a panoramic picture of a video surveillance frame at a time corresponding to the capturing rule includes:
[0025] If the time for triggering the alarm rule reaches a preset time, capturing a panoramic picture of the video surveillance frame corresponding to the preset time; and / or
[0026] If the target object cannot be identified in one of the plurality of consecutive video surveillance frames, and the target object is identified in a previous video surveillance frame of the one video surveillance frame, a panoramic image of the previous video surveillance frame is captured.
[0027] In this embodiment, if the time when the alarm rule is triggered reaches a preset time, a panoramic picture of the video surveillance frame corresponding to the preset time is captured, and / or if the target object cannot be identified in a video surveillance frame of a plurality of consecutive video surveillance frames, and the target object is identified in a video surveillance frame before a video surveillance frame, a panoramic picture of the previous video surveillance frame is captured. Since the capture rule is that the time when the alarm rule is triggered reaches a preset time, and / or before the target object disappears in the video surveillance frame, it can be ensured that when the panoramic picture of the video surveillance frame corresponding to the capture rule is captured, the target object exists in the video surveillance frame.
[0028] In a second aspect, the present application also provides an image display method, the method comprising:
[0029] Receive a panoramic picture sent by a video surveillance device and a recognition result of a video surveillance frame corresponding to the panoramic picture, wherein the panoramic picture is obtained by capturing the last video surveillance frame of a plurality of consecutive video surveillance frames after a video surveillance frame in which a target object that triggers an alarm rule exists, and the recognition result is determined after recognizing the target object in the last video surveillance frame;
[0030] A target frame including the target object is extracted from the panoramic picture according to the recognition result, and the panoramic picture and the target frame are displayed.
[0031] The data image display method provided in the embodiment of the present application can receive a panoramic picture sent by a video surveillance device and the recognition result of a video surveillance frame corresponding to the panoramic picture, wherein the panoramic picture is obtained by capturing the last video surveillance frame in a plurality of consecutive video surveillance frames after a video surveillance frame in which a target object that triggers an alarm rule exists, and the recognition result is determined after identifying the target object in the last video surveillance frame, and a target frame including the target object is extracted from the panoramic picture according to the recognition result, and the panoramic picture and the target frame are displayed. Since only one panoramic picture needs to be captured, and the target frame including the target object that triggers the alarm rule is captured in the panoramic picture, the storage space required for storing pictures can be reduced, and the storage pressure of related equipment can be reduced.
[0032] In a third aspect, an embodiment of the present application further provides a device for capturing an image, comprising:
[0033] A target object recognition unit, configured to recognize the target object in a plurality of subsequent consecutive video surveillance frames if a target object that triggers an alarm rule exists in the video surveillance frame;
[0034] An image capture unit is used to capture a panoramic image of a video surveillance frame that meets the capture rule at a corresponding time after the capture rule is met; wherein the video surveillance frame that meets the capture rule at a corresponding time is the last video surveillance frame among the continuous multiple video surveillance frames; and the continuous multiple video surveillance frames all include the target object;
[0035] The image sending unit is used to send the recognition result of the panoramic picture and the video surveillance frame at the time corresponding to the capture rule to the back end, so that the back end extracts the target frame including the target object in the panoramic picture according to the recognition result.
[0036] In an optional embodiment, the target object identification unit is specifically used to:
[0037] If there is a target object that triggers the alarm rule in the video surveillance frame, the target identifier and position coordinates corresponding to the target object are determined respectively; the position coordinates corresponding to the target object are determined according to the first position of the target object in the video surveillance frame;
[0038] According to the target identifier corresponding to the target object, the target object is identified in a plurality of subsequent consecutive video surveillance frames, and a second position of the target object in the plurality of consecutive video surveillance frames is determined respectively;
[0039] Each time a second position is determined, the position coordinates are updated according to the determined second position.
[0040] In an optional embodiment, the target object identification unit is further used to:
[0041] For each of the plurality of consecutive video surveillance frames, perform the following operations respectively:
[0042] Identify the video surveillance frame to determine each object in the video surveillance frame;
[0043] According to the target identifier corresponding to the target object, the target object is matched with each of the objects to determine a second position of the target object in the video surveillance frame.
[0044] In an optional embodiment, the image sending unit is specifically configured to:
[0045] The target identifier and position coordinates corresponding to the target object are sent to the back end; the position coordinates corresponding to the target object are determined according to the second position of the target object in the video surveillance frame at the time corresponding to the capture rule.
[0046] In an optional embodiment, the image capture unit is specifically used to:
[0047] If the time for triggering the alarm rule reaches a preset time, capturing a panoramic picture of the video surveillance frame corresponding to the preset time; and / or
[0048] If the target object cannot be identified in one of the plurality of consecutive video surveillance frames, and the target object is identified in a previous video surveillance frame of the one video surveillance frame, a panoramic image of the previous video surveillance frame is captured.
[0049] In a fourth aspect, the present application also provides an image display device, including:
[0050] An image receiving unit is used to receive a panoramic picture sent by a video surveillance device and a recognition result of a video surveillance frame corresponding to the panoramic picture, wherein the panoramic picture is obtained by capturing the last video surveillance frame of a plurality of consecutive video surveillance frames after a video surveillance frame in which a target object that triggers an alarm rule exists, and the recognition result is determined after recognizing the target object in the last video surveillance frame;
[0051] An image display unit is used to extract a target frame including a target object from the panoramic picture according to the recognition result, and to display the panoramic picture and the target frame.
[0052] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for capturing an image of the first aspect is implemented.
[0053] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image display method of the second aspect is implemented.
[0054] In the seventh aspect, an embodiment of the present application also provides a video surveillance device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the processor implements the method for capturing images of the first aspect.
[0055] In an eighth aspect, an embodiment of the present application further provides an image display device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the processor implements the image display method of the second aspect.
[0056] The technical effects brought about by any one of the implementation methods in the third, fifth and seventh aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here.
[0057] The technical effects brought about by any one of the implementation methods in the fourth, sixth and eighth aspects can refer to the technical effects brought about by the corresponding implementation method in the second aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0059] Figure 1 A schematic diagram of the structure of an image processing system provided in an embodiment of the present application;
[0060] Figure 2 A schematic diagram of the structure of a video surveillance device provided in an embodiment of the present application;
[0061] Figure 3 A schematic diagram of the structure of another image display device provided in an embodiment of the present application;
[0062] Figure 4 A flowchart of a method for capturing an image provided in an embodiment of the present application;
[0063] Figure 5 A flowchart of another method for capturing an image provided in an embodiment of the present application;
[0064] Figure 6 A flowchart of an image display method provided in an embodiment of the present application;
[0065] Figure 7 A flowchart of another image display method provided in an embodiment of the present application;
[0066] Figure 8 A schematic diagram of a video surveillance frame provided in an embodiment of the present application;
[0067] Fig. 9A schematic diagram of an image capture result provided in an embodiment of the present application;
[0068] Fig.10 A schematic diagram of the structure of a device for capturing images provided in an embodiment of the present application;
[0069] Fig.11 A schematic diagram of the structure of an image display device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0070] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0071] It should be noted that the terms "including" and "having" and their variations involved in the documents of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0072] The technical solution provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0073] The word "exemplary" used hereinafter means "serving as an example, embodiment, or illustrative". Any embodiment described as "exemplary" is not necessarily to be interpreted as superior or better than other embodiments. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.
[0074] Figure 1 A schematic diagram of the structure of an image processing system is shown in FIG. Figure 1 As shown, the image processing system may include a video monitoring device 100 and an image display device 200 .
[0075] The video surveillance device 100 may be any device capable of implementing the method for capturing images proposed in the present application. For example, the video surveillance device 100 may be a camera. In this embodiment, the structure of the video surveillance device 100 may be as follows: Figure 2 As shown, it includes a memory 101 , a sending component 103 and one or more processors 102 .
[0076] The memory 101 is used to store computer programs executed by the processor 102. The memory 101 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.
[0077] The memory 101 may be a volatile memory, such as a random-access memory (RAM); the memory 101 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 101 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 101 may be a combination of the above memories.
[0078] The processor 102 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 102 is used to implement the method for capturing an image provided in the embodiment of the present application when calling the computer program stored in the memory 101 .
[0079] The sending component 103 is used to send the panoramic picture and the recognition result of the video surveillance frame at the time corresponding to the capture rule to the image display device 200.
[0080] The specific connection medium between the memory 101, the sending component 103 and the processor 102 is not limited in the embodiment of the present application. Figure 2 In the embodiment, the memory 101 and the processor 102 are connected via a bus 104. The bus 104 is Figure 2 The connections between other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus 104 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0081] The image display device 200 may be any device capable of implementing the image display method proposed in the present application, and the image display device 200 may be a backend. For example, the image display device 200 may be an NVR (Network Video Recorder), a DVR (Digital Video Recorder), an IVSS (Intelligent Video Surveillance System), a platform, etc. In this embodiment, the structure of the image display device 200 may be as follows: Figure 3 As shown, it includes a memory 201 , a receiving component 203 and one or more processors 202 .
[0082] The memory 201 is used to store computer programs executed by the processor 202. The memory 201 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.
[0083] The memory 201 may be a volatile memory, such as a random-access memory (RAM); the memory 201 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 201 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 201 may be a combination of the above memories.
[0084] The processor 202 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 202 is used to implement the image display method provided in the embodiment of the present application when calling the computer program stored in the memory 201 .
[0085] The receiving component 203 is used to receive the panoramic picture sent by the video surveillance device 100 and the recognition result of the corresponding video surveillance frame.
[0086] The specific connection medium between the memory 201, the receiving component 203 and the processor 202 is not limited in the embodiment of the present application. Figure 3In the embodiment, the memory 201 and the processor 202 are connected via a bus 204. The bus 204 is connected to the processor 202 via a bus 204. Figure 3 The connections between other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus 204 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0087] The video surveillance device 100 may be directly connected to the image display device 200 or be communicatively connected via a network, or may be communicatively connected via other means, which is not limited in the present application.
[0088] In some embodiments, the flowchart of the method for capturing images executed by the video surveillance device 100 can be found in Figure 4 As shown, the following steps are included:
[0089] Step S401: If there is a target object that triggers an alarm rule in a video surveillance frame, the target object is identified in a plurality of subsequent consecutive video surveillance frames.
[0090] If there is a target object that triggers an alarm rule in the video surveillance frame, the target identifier and position coordinates corresponding to the target object can be determined respectively. The position coordinates corresponding to the target object are determined based on the first position of the target object in the video surveillance frame. Alarm rules may include line tripping rules, area intrusion rules, and item left behind rules.
[0091] Then, for each of the multiple consecutive video surveillance frames, the following operations can be performed respectively: identify the video surveillance frame, determine each object in the video surveillance frame, match the target object with each object according to the target identifier corresponding to the target object, and determine the second position of the target object in the video surveillance frame.
[0092] Furthermore, each time a second position is determined, the position coordinates corresponding to the target object may be updated according to the determined second position.
[0093] Step S402, after the capture rule is met, a panoramic picture of the video surveillance frame that meets the capture rule at the corresponding time is captured.
[0094] The video surveillance frame satisfying the corresponding time of the capture rule is the last video surveillance frame among a plurality of consecutive video surveillance frames, and the plurality of consecutive video surveillance frames all include the target object.
[0095] In one embodiment, if the time of triggering the alarm rule reaches a preset time, a panoramic picture of a video surveillance frame corresponding to the preset time is captured, and the video surveillance frame corresponding to the preset time includes the target object.
[0096] In another embodiment, if the target object cannot be identified in a video surveillance frame of a plurality of consecutive video surveillance frames, and the target object is identified in a video surveillance frame before the video surveillance frame, a panoramic image of the previous video surveillance frame is captured.
[0097] In another embodiment, if the time for triggering the alarm rule reaches a preset time, and the target object cannot be identified in a video surveillance frame of multiple consecutive video surveillance frames, and the target object is identified in a video surveillance frame before a video surveillance frame, a panoramic picture of the video surveillance frame corresponding to the current moment is captured.
[0098] Step S403, sending the recognition result of the panoramic picture and the video surveillance frame at the time corresponding to the capture rule to the back end, so that the back end extracts the target frame including the target object in the panoramic picture according to the recognition result.
[0099] The panoramic picture, the target identifier and the position coordinates corresponding to the target object can be sent to the back end. The panoramic picture is obtained by capturing the video surveillance frame that meets the corresponding time of the capture rule, and the position coordinates corresponding to the target object are determined according to the second position of the target object in the video surveillance frame at the corresponding time of the capture rule.
[0100] The method for capturing images provided in the embodiment of the present application, when a target object that triggers an alarm rule is detected in a video surveillance frame, that is, when an event occurs, does not immediately capture a panoramic picture of the video surveillance frame, but delays N milliseconds, that is, determines whether the capture rule is met, and identifies the target object in multiple consecutive video surveillance frames included in the N milliseconds, that is, within the N milliseconds, each video surveillance frame will update the position coordinates of the target object according to the target identifier corresponding to the target object, that is, all target objects where an event occurs within N milliseconds are mapped to the Mth video surveillance frame corresponding to N milliseconds, then when the Nth millisecond is reached, the Mth video surveillance frame corresponding to the Nth millisecond is captured, and the target identifiers and position coordinates of all target objects corresponding to the Mth video surveillance frame are sent to the back end. Since the relative accuracy of the positions of all target objects in the Mth video surveillance frame can be guaranteed, the backend can accurately capture the target frames including all target objects according to the Mth video surveillance frame. In addition, since only the Mth video surveillance frame is captured, the storage space required for storing images can be reduced, so as to solve the problem of not being able to capture all target objects in real time when the performance of related equipment is insufficient.
[0101] The calculation formula of the above crawling rule N can be shown as follows:
[0102]
[0103] Among them, the value range of N is N greater than or equal to 0, and when N is equal to 0, it means that the capture rule is met and the video surveillance frame is captured immediately. Cs is the intelligent detection frame rate, which is the actual frame rate configured by the device for intelligent detection of video surveillance frames. Different values can be configured according to the device and business needs. For example, it can be configured to 8, 12, 16, 24, etc. Cp is the comprehensive evaluation index of the device hardware, with a value range of Cp = {0, 1}. It is obtained based on the CPU main frequency Qa (unit: MHZ) of the device main control, the number of CPUs Qn, the bandwidth Qbw (unit: GB / s), the computing power Qt (unit: TOPS) and other indicators. The technical formula is: Cp = (Qa / 900)*Qn+Qbw / 3.6+Qt / 0.5. Cf is the comprehensive evaluation coefficient of the device performance index, which can be determined based on the maximum number of events Mf of a single video surveillance frame and the maximum number of alarms per second Mt: Cf = (Mf+Mt) / 16. Ct is the time consumed for intelligent processing of each frame, which is the time consumed for intelligently processing a video surveillance frame data in the current operating environment, and this parameter can be calculated and obtained when the device is running.
[0104] Nt is the time from when the target object triggers the alarm rule in one video surveillance frame to when the target object disappears in another video surveillance frame, when the target object disappears within N milliseconds after triggering the alarm rule.
[0105] In some other embodiments, the flowchart of the method for capturing images executed by the video surveillance device 100 can also be found in Figure 5 As shown, the following steps are included:
[0106] Step S501: Detect the video surveillance frame.
[0107] Each object in the video surveillance frame can be detected in real time according to the set alarm rules.
[0108] Step S502, determining whether there is a target object in the video surveillance frame that triggers an alarm rule; if so, executing step S503; if not, executing step S504.
[0109] Step S503: Add the target identifier and position coordinates corresponding to the target object to the linked list.
[0110] Determine whether there is a target object in the video surveillance frame that triggers the alarm rule. If so, the target identifier and position coordinates corresponding to the target object can be obtained, where the position coordinates corresponding to the target object are determined based on the position of the target object in the video surveillance frame, and the target identifier and position coordinates corresponding to the target object are added to the linked list.
[0111] Step S504, determining whether there is a target object that triggers the alarm rule in the video surveillance frame; if so, executing step S505; if not, executing step S501.
[0112] Step S505: update the position coordinates corresponding to the target object in the linked list.
[0113] After the target identifier and position coordinates corresponding to the target object are added to the linked list, the position coordinates corresponding to the target object in the linked list may be updated according to the position of the target object in the next video surveillance frame.
[0114] If there is no target object in the video surveillance frame that triggers the alarm rule, it can be determined whether there is a target object in the video surveillance frame that triggers the alarm rule. If there is, the position coordinates corresponding to the target object in the linked list can be updated according to the position of the target object in the video surveillance frame. If there is no target object in the video surveillance frame, the next video surveillance frame can be tested to determine whether there is a target object in the next video surveillance frame that triggers the alarm rule.
[0115] Specifically, the process of updating the position coordinates corresponding to the target object in the linked list can be: obtaining each object in the video surveillance frame, and matching the target identifier corresponding to the target object in the linked list with each object respectively, and determining whether the target identifier corresponding to the target object is the same as each object. If they are the same, the position coordinates corresponding to the target object in the linked list can be updated according to the position of the target object in the video surveillance frame.
[0116] Step S506, determine whether the crawling rules are met; if so, execute step S507; if not, execute step S501.
[0117] Step S507, capturing a panoramic picture of the video surveillance frame at the time corresponding to the capture rule.
[0118] It can be judged whether the capture rules are met, that is, whether the time to trigger the alarm rule reaches the preset time, and / or whether there is a situation where the target object cannot be detected in a video surveillance frame, but the target object can be detected in the previous video surveillance frame of the video surveillance frame, that is, whether the target object disappears prematurely.
[0119] If the capture rule is met, a panoramic picture of the video surveillance frame at the time corresponding to the capture rule can be captured.
[0120] If the capture rule is not satisfied, the video surveillance frame may continue to be detected to determine whether there is a target object in the video surveillance frame that triggers an alarm rule.
[0121] Step S508: Send the panoramic picture, the target identifier and the position coordinates corresponding to the target object to the back end.
[0122] The panoramic picture of the video surveillance frame that meets the corresponding time of the capture rule, as well as the target identifier and position coordinates corresponding to the target object in the linked list can be sent to the back end, wherein the position coordinates are based on the position of the target object in the video surveillance frame at the corresponding time of the capture rule.
[0123] After the panoramic image, the target identifier and the position coordinates corresponding to the target object are sent to the backend, the linked list can be cleared.
[0124] In some embodiments, the flowchart of the image display method performed by the image display device 200 can be found in Figure 6 As shown, the following steps are included:
[0125] Step S601: receiving a panoramic picture sent by a video surveillance device and a recognition result of a video surveillance frame corresponding to the panoramic picture.
[0126] The panoramic image is obtained by capturing the last video surveillance frame of a plurality of consecutive video surveillance frames after the video surveillance frame in which the target object that triggers the alarm rule exists, and the recognition result is determined after the target object in the last video surveillance frame is recognized. The recognition result is the position coordinates corresponding to the target object and the target identifier corresponding to the target object determined according to the position of the target object in the last video surveillance frame.
[0127] Step S602: extract a target frame including the target object from the panoramic picture according to the recognition result, and display the panoramic picture and the target frame.
[0128] According to the target identifier and position coordinates corresponding to the target object, a target frame including the target object is extracted from the panoramic image, and in response to a user instruction to display the image, the panoramic image and the target frame can be displayed in a related interface.
[0129] In some other embodiments, the flowchart of the image display method executed by the image display device 200 can also be found in Figure 7 As shown, the following steps are included:
[0130] Step S701: receiving a panoramic picture sent by a video surveillance device and a recognition result of a video surveillance frame corresponding to the panoramic picture.
[0131] The panoramic image is a panoramic image of a video surveillance frame that satisfies the corresponding time of the capture rule, and the recognition result includes the position coordinates corresponding to the target object and the target identifier corresponding to the target object determined according to the position of the target object in the video surveillance frame that satisfies the corresponding time of the capture rule.
[0132] Step S702: extracting the attributes and feature information of the target object according to the recognition result.
[0133] According to the position coordinates and target identifier corresponding to the target object, the attributes and feature information of the target object can be extracted from the panoramic image, and the extracted attributes and feature information can be analyzed.
[0134] Step S703, determining whether to display a target frame including the target object in the panoramic picture; if so, executing step S704; if not, executing step S705.
[0135] Step S704: Draw a target frame including the target object in the panoramic picture according to the recognition result.
[0136] In response to a user instruction to display a target frame including a target object in a panoramic picture, a target frame including the target object may be drawn in the panoramic picture according to the position coordinates and the target identifier corresponding to the target object.
[0137] Step S705, determine whether to display the target frame alone; if so, execute step S706; if not, execute step S707.
[0138] Step S706, cropping the target frame.
[0139] In response to the user's instruction to display the target frame alone, after the target frame including the target object is drawn in the panoramic picture, the target frame may be cropped.
[0140] Step S707, displaying the panoramic image, target frame, and properties and feature information of the target object in the interface.
[0141] After a target frame including the target object is drawn in the panoramic image, the attributes and feature information of the panoramic image, the target frame, and the target object may be displayed in a related interface.
[0142] Step S708, displaying the target frame in the interface.
[0143] After the target frame is obtained by cropping, the target frame can be displayed in the relevant interface.
[0144] For example, Figure 8 Multiple video surveillance frames are shown. It is assumed that the target object ID1 triggers the alarm rule in the first frame, and the target object ID2 triggers the alarm rule in the second frame.
[0145] In the first frame, the target object ID1 is detected to trigger the alarm rule. At this time, the corresponding object of the first frame is not captured, but the position coordinates determined according to the position of the target object ID1 in the first frame are saved in the linked list, and the time starts counting.
[0146] In the second frame, the target object ID2 triggers the alarm rule, and the position coordinates determined according to the position of the target object ID2 in the second frame can be saved in the linked list, and the position coordinates of the target object ID1 saved in the linked list are updated to the position coordinates determined according to the position of the target object ID1 in the second frame, and the time count is fixedly accumulated.
[0147] The same process as the second frame is performed in the consecutive multiple frames after the second frame, that is, in each frame of the consecutive multiple frames after the second frame, it is detected whether there is a target object that triggers the alarm rule. If so, the position coordinates corresponding to the target object are saved in the linked list, and the position coordinates of the target object ID1 and the target object ID2 saved in the linked list are updated according to the positions of the target object ID1 and the target object ID2 in the frame.
[0148] When the time reaches the capture rule N milliseconds or one of the target objects ID1 and ID2 disappears in advance, the current frame is captured, that is, the panoramic image of the Mth frame corresponding to the Nth millisecond is captured. The captured panoramic image is Figure 8 Then, the target identifier and position coordinates corresponding to the panoramic image, target object ID1 and target object ID2 are sent to the back end.
[0149] After receiving the target identifiers and position coordinates corresponding to the panoramic image, target object ID1 and target object ID2, the backend can extract the target frame including the target object ID1 and the target frame including the target object ID2 from the panoramic image according to the position coordinates of the target object ID1 and the position coordinates of the target object ID2. The obtained images can be respectively as follows: Fig. 9 (a) and Fig. 9 (b) as shown.
[0150] and Figure 4 The method for capturing an image shown is based on the same inventive concept, and an apparatus for capturing an image is also provided in the embodiment of the present application. Since the apparatus is the apparatus corresponding to the method for capturing an image of the present application, and the principle of solving the problem by the apparatus is similar to that of the method, the implementation of the apparatus can refer to the implementation of the above method, and the repeated parts will not be repeated.
[0151] Fig.10 A schematic diagram of the structure of a device for capturing an image provided by an embodiment of the present application is shown. Fig.10 As shown, the device for capturing images includes a target object recognition unit 1001 , an image capturing unit 1002 and an image sending unit 1003 .
[0152] The target object identification unit 1001 is used to identify the target object in a plurality of subsequent consecutive video surveillance frames if a target object that triggers an alarm rule exists in the video surveillance frame;
[0153] The image capture unit 1002 is used to capture a panoramic image of a video surveillance frame that meets the capture rule at a corresponding time after the capture rule is met; wherein the video surveillance frame that meets the capture rule at a corresponding time is the last video surveillance frame among a plurality of consecutive video surveillance frames; and the plurality of consecutive video surveillance frames all include the target object;
[0154] The image sending unit 1003 is used to send the recognition result of the panoramic picture and the video surveillance frame at the time corresponding to the capture rule to the back end, so that the back end extracts the target frame including the target object in the panoramic picture according to the recognition result.
[0155] In an optional embodiment, the target object identification unit 1001 is specifically configured to:
[0156] If there is a target object that triggers the alarm rule in the video surveillance frame, the target identifier and position coordinates corresponding to the target object are determined respectively; the position coordinates corresponding to the target object are determined according to the first position of the target object in the video surveillance frame;
[0157] According to the target identifier corresponding to the target object, the target object is identified in a plurality of subsequent consecutive video surveillance frames, and a second position of the target object in the plurality of consecutive video surveillance frames is determined respectively;
[0158] Each time a second position is determined, the position coordinates are updated according to the determined second position.
[0159] In an optional embodiment, the target object identification unit 1001 is further configured to:
[0160] For each of the multiple consecutive video surveillance frames, perform the following operations respectively:
[0161] Identify the video surveillance frame and determine each object in the video surveillance frame;
[0162] According to the target identifier corresponding to the target object, the target object is matched with each object to determine the second position of the target object in the video surveillance frame.
[0163] In an optional embodiment, the image sending unit 1003 is specifically configured to:
[0164] The target identifier and position coordinates corresponding to the target object are sent to the back end; the position coordinates corresponding to the target object are determined according to the second position of the target object in the video surveillance frame at the time corresponding to the capture rule.
[0165] In an optional embodiment, the image capture unit 1002 is specifically configured to:
[0166] If the time to trigger the alarm rule reaches a preset time, a panoramic image of the video surveillance frame corresponding to the preset time is captured; and / or
[0167] If the target object cannot be identified in one of the plurality of consecutive video surveillance frames, and the target object is identified in a video surveillance frame preceding the one of the video surveillance frames, a panoramic image of the preceding video surveillance frame is captured.
[0168] and Figure 6 The image display method shown is based on the same inventive concept, and an image display device is also provided in the embodiment of the present application. Since the device is a device corresponding to the image display method of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the above method, and the repeated parts will not be repeated.
[0169] Fig.11 A schematic diagram of the structure of an image display device provided in an embodiment of the present application is shown. Fig.11 As shown, the image display device includes an image receiving unit 1101 and an image display unit 1102 .
[0170] The image receiving unit 1101 is used to receive the panoramic picture sent by the video surveillance device and the recognition result of the video surveillance frame corresponding to the panoramic picture, wherein the panoramic picture is obtained by capturing the last video surveillance frame of a plurality of consecutive video surveillance frames after the video surveillance frame in which the target object that triggers the alarm rule exists, and the recognition result is determined after recognizing the target object in the last video surveillance frame;
[0171] The image display unit 1102 is used to extract a target frame including the target object from the panoramic image according to the recognition result, and display the panoramic image and the target frame.
[0172] According to one aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method of capturing an image or the image display method in the above-mentioned embodiment. The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0173] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.
Claims
1. A method for capturing an image, characterized in that: The method comprises: If there is a target object that triggers the alarm rule in the video surveillance frame, the target object is identified in a plurality of subsequent consecutive video surveillance frames; After the capture rule is satisfied, a panoramic picture corresponding to the video surveillance frame satisfying the capture rule corresponding to the time is captured; wherein the capture rule represents the time for delaying the capture of the panoramic picture after the alarm rule is triggered, and the capture rule is determined according to the performance of the video surveillance device, or the time when the target object disappears after the alarm rule is triggered; the video surveillance frame satisfying the capture rule corresponding to the time is the last video surveillance frame among the multiple consecutive video surveillance frames; the multiple consecutive video surveillance frames all include the target object; The recognition result of the panoramic picture and the video surveillance frame at the time corresponding to the capture rule is sent to the back end, so that the back end extracts the target frame including the target object in the panoramic picture according to the recognition result.
2. The method according to claim 1, characterized in that If there is a target object that triggers an alarm rule in the video surveillance frame, identifying the target object in a plurality of subsequent consecutive video surveillance frames includes: If there is a target object that triggers the alarm rule in the video surveillance frame, the target identifier and position coordinates corresponding to the target object are determined respectively; the position coordinates corresponding to the target object are determined according to the first position of the target object in the video surveillance frame; According to the target identifier corresponding to the target object, the target object is identified in a plurality of subsequent consecutive video surveillance frames, and a second position of the target object in the plurality of consecutive video surveillance frames is determined respectively; Each time a second position is determined, the position coordinates are updated according to the determined second position.
3. The method according to claim 2, characterized in that The step of identifying the target object in a plurality of subsequent consecutive video surveillance frames according to the target identifier corresponding to the target object, and determining a second position of the target object in the plurality of consecutive video surveillance frames respectively, includes: For each of the plurality of consecutive video surveillance frames, perform the following operations respectively: Identify the video surveillance frame to determine each object in the video surveillance frame; According to the target identifier corresponding to the target object, the target object is matched with each of the objects to determine a second position of the target object in the video surveillance frame.
4. The method according to claim 2, characterized in that: The step of sending the recognition result of the video surveillance frame at the time corresponding to the capture rule to the back end includes: The target identifier and position coordinates corresponding to the target object are sent to the back end; the position coordinates corresponding to the target object are determined according to the second position of the target object in the video surveillance frame at the time corresponding to the capture rule.
5. The method according to any one of claims 1 to 4, characterized in that: The capturing of the panoramic picture of the video surveillance frame at the time corresponding to the capturing rule includes: If the time for triggering the alarm rule reaches a preset time, capturing a panoramic picture of the video surveillance frame corresponding to the preset time; and / or If the target object cannot be identified in one of the plurality of consecutive video surveillance frames, and the target object is identified in a previous video surveillance frame of the one video surveillance frame, a panoramic image of the previous video surveillance frame is captured.
6. An image display method, characterized in that: The method comprises: Receive a panoramic picture sent by a video surveillance device and a recognition result of a video surveillance frame corresponding to the panoramic picture, wherein the panoramic picture is obtained by capturing the last video surveillance frame of a plurality of consecutive video surveillance frames after a video surveillance frame in which a target object that triggers an alarm rule exists after a capture rule is satisfied, and the recognition result is determined after identifying the target object in the last video surveillance frame; the capture rule represents a time for delaying the capture of the panoramic picture after the alarm rule is triggered, and the capture rule is determined based on the performance of the video surveillance device, or the time when the target object disappears after the alarm rule is triggered; A target frame including the target object is extracted from the panoramic picture according to the recognition result, and the panoramic picture and the target frame are displayed.
7. A device for capturing an image, characterized in that: include: A target object recognition unit, configured to recognize the target object in a plurality of subsequent consecutive video surveillance frames if a target object that triggers an alarm rule exists in the video surveillance frame; An image capture unit is used to capture a panoramic image of a video surveillance frame that meets the corresponding time of the capture rule after the capture rule is met; wherein the capture rule represents the time for delaying the capture of the panoramic image after the alarm rule is triggered, and the capture rule is determined according to the performance of the video surveillance device, or the time when the target object disappears after the alarm rule is triggered; the video surveillance frame that meets the corresponding time of the capture rule is the last video surveillance frame among the multiple consecutive video surveillance frames; the multiple consecutive video surveillance frames all include the target object; The image sending unit is used to send the recognition result of the panoramic picture and the video surveillance frame at the time corresponding to the capture rule to the back end, so that the back end extracts the target frame including the target object in the panoramic picture according to the recognition result.
8. An image display device, characterized in that: include: An image receiving unit is used to receive a panoramic picture sent by a video surveillance device and a recognition result of a video surveillance frame corresponding to the panoramic picture, wherein the panoramic picture is obtained by capturing the last video surveillance frame of a plurality of consecutive video surveillance frames after a video surveillance frame in which a target object that triggers an alarm rule exists after a capture rule is satisfied, and the recognition result is determined after identifying the target object in the last video surveillance frame; the capture rule represents a time for delaying the capture of the panoramic picture after the alarm rule is triggered, and the capture rule is determined based on the performance of the video surveillance device, or the time when the target object disappears after the alarm rule is triggered; An image display unit is used to extract a target frame including a target object from the panoramic picture according to the recognition result, and to display the panoramic picture and the target frame.
9. A video surveillance device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the method according to any one of claims 1 to 5 is implemented.
10. An image display device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the method described in claim 6 is implemented.
11. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 5 or claim 6 is implemented.
