Event snapshot method, device and system of camera and computer equipment

Abstract

The invention relates to an event snapshot method, device and system of a camera and computer equipment. The method comprises the following steps: in a process of analyzing a monitoring picture image of a camera, generating a new snapshot task from an event which is generated by analysis and needs to be snapshot, inserting the new snapshot task into a snapshot task queue, and snapshot the snapshot task in the snapshot task queue by using the camera, sending the snapshot tasks which cannot be inserted into the snapshot task queue to a rear-end processor connected with the camera for snapshot; the length of the snapshot task queue is associated with the hardware performance of the camera and the current network flow. By adopting the method, the problems of image loss / event loss in the snapshot process caused by a large number of snapshot tasks generated in a short time and network lagging or packet loss in the snapshot data transmission process can be solved.

Description

Technical Field

[0001] The present application relates to the field of security technology, and in particular to a method, device, system and computer equipment for capturing an event using a camera. Background Art

[0002] With the development of electronic technology, security monitoring technology has also developed rapidly. In the process of security monitoring, the monitoring system will automatically trigger the capture task according to the preset rules or algorithms, capture certain events, such as the target crossing the line or the target entering / leaving the preset area, and send the captured pictures or videos of the events to the server that subscribes to the events.

[0003] Event capture is usually achieved by using the hardware encoder of the surveillance camera, but the number of hardware encoders of surveillance cameras is limited. When a large number of events that need to be captured occur in the surveillance area in a short period of time and a large number of capture tasks are generated, the number of capture tasks may exceed the processing capacity of the hardware encoder of the surveillance camera, and there may be a situation of lost images / events due to task accumulation. In addition, a large number of capture tasks in a short period of time may generate a large amount of data, and the data transmission needs to be achieved through the network. If the amount of data generated in a short period of time exceeds the transmission capacity of the network, there may be a situation of network jamming or packet loss due to a large amount of bandwidth being occupied.

[0004] However, for the event capture method of existing cameras, since a large number of capture tasks are generated in a short period of time, there are problems such as lost images / events during the capture process and network lag or packet loss during the transmission of capture data. Currently, no effective solution has been proposed. Summary of the invention

[0005] Based on this, it is necessary to provide a camera event capture method, device, system and computer equipment to address the above technical problems.

[0006] In a first aspect, the present application provides a method for capturing an event using a camera. The method comprises the following steps:

[0007] In the process of analyzing the camera's surveillance screen images, a new snapshot task is generated based on the events that need to be captured, the new snapshot task is inserted into a snapshot task queue, and the camera is used to capture the snapshot tasks in the snapshot task queue. The snapshot tasks that cannot be inserted into the snapshot task queue are sent to a back-end processor connected to the camera for capture; the length of the snapshot task queue is related to the hardware performance of the camera and the current network traffic.

[0008] In one of the embodiments, in the process of analyzing the surveillance screen image of the camera, a new capture task is generated by the analysis of the event that needs to be captured, the new capture task is inserted into the capture task queue, and the capture tasks in the capture task queue are captured by the camera, and the capture tasks that cannot be inserted into the capture task queue are sent to the back-end processor connected to the camera for capture, including the following steps:

[0009] Performing algorithm analysis on the surveillance screen image of the camera at a preset frame rate, identifying the target event in the surveillance screen image, and generating a capture task of the target event based on the identification result of the target event;

[0010] The capture task of the target event is inserted into the capture task queue according to the priority of the preset event type, and the camera is used to capture the capture task in the capture task queue to obtain the first image data of the capture task inserted into the capture task queue, and the capture task that cannot be inserted into the capture task queue is converted into a capture task event, and sent to the back-end processor connected to the camera for capture, to obtain the second image data of the capture task that cannot be inserted into the capture task queue; the capture task event includes the index correspondence between the image to be captured of the capture task and the video code stream of the storage server, and the OSD overlay information of the capture task.

[0011] In one embodiment, after the capture task of the target event is inserted into the capture task queue according to the priority of the preset event type, and the capture task in the capture task queue is captured by the camera to obtain the first image data of the capture task inserted into the capture task queue, the following steps are included:

[0012] Based on the preset priority of the business data, the first image data is transmitted to the back-end processor connected to the camera for storage.

[0013] In one of the embodiments, the capture task of the target event is inserted into the capture task queue according to the priority of the preset event type, and the capture task in the capture task queue is captured by the camera to obtain the first image data of the capture task inserted into the capture task queue, and the capture task that cannot be inserted into the capture task queue is converted into a capture task event, and sent to the back-end processor connected to the camera for capture, so as to obtain the second image data of the capture task that cannot be inserted into the capture task queue, including the following steps:

[0014] When the snapshot task queue is idle, based on the preset event type priority, inserting the snapshot task of the target event into the snapshot task queue, and using the camera to capture the snapshot tasks in the snapshot task queue to obtain the first image data;

[0015] When the snapshot task queue is not idle and the priority of the event type of the target event is higher than the priority of the event type corresponding to the snapshot task at the end of the snapshot task queue, based on the preset event type priority, the snapshot task of the target event is inserted into the snapshot task queue, and the snapshot tasks in the snapshot task queue are captured by the camera to obtain the first data with images, and the snapshot tasks squeezed out of the snapshot task queue are converted into the snapshot task events, which are sent to the back-end processor for capturing to obtain the second data with images;

[0016] When the snapshot task queue is full and the priority of the event type of the target event is lower than or equal to the priority of the event type corresponding to the snapshot task at the end of the snapshot task queue, the snapshot task of the target event is converted into the snapshot task event, sent to the back-end processor for snapshot, and the second image data is obtained.

[0017] In one embodiment, the event capture method of the camera further includes the following steps:

[0018] Based on the network traffic monitoring result, the length of the snapshot task queue is adjusted.

[0019] In one embodiment, adjusting the length of the snapshot task queue based on the network traffic monitoring result includes the following steps:

[0020] Determine the current available bandwidth based on the network bandwidth and the current upstream traffic;

[0021] When the current available bandwidth is continuously less than the preset first bandwidth threshold within a preset time interval, or the bandwidth margin of the snapshot task queue after the snapshot task traffic application is less than the preset second bandwidth threshold, and the amount of data to be sent in the snapshot task queue is greater than the preset data amount threshold, the length of the snapshot task queue is shortened.

[0022] In a second aspect, the present application also provides an event capture device. The device includes a camera end and a backend processor;

[0023] The camera end is used to determine the target event in response to the subscription information sent by the back-end processor; the camera end is also used to adjust the length of the capture task queue based on the network traffic monitoring result; the camera end is also used to perform algorithm analysis on the camera's monitoring screen image at a preset frame rate, identify the target event in the monitoring screen image, and generate a capture task of the target event based on the recognition result of the target event; the capture task of the target event is preferentially inserted into the capture task queue according to the priority of the event type, and the capture task in the capture task queue is captured by the camera to obtain the first image data of the capture task that has been inserted into the capture task queue, and the capture task that cannot be inserted into the capture task queue is converted into a capture task event, and sent to the back-end processor for capture, to obtain the second image data of the capture task that cannot be inserted into the capture task queue; the camera end is also used to transmit the first image data to the back-end processor for storage in the order of the business data priority from high to low based on the preset business data priority;

[0024] The back-end processor is used to generate the subscription information according to the subscription event set by the user, and send the subscription information to the camera end; receive the first image data of the snapshot task sent by the camera end, and store the received first image data; the back-end processor is also used to receive the snapshot task event sent by the camera end, and based on the received snapshot task event, determine the video frame corresponding to the snapshot task stored in the storage server; based on the video frame corresponding to the snapshot task stored in the storage server, capture the snapshot task and restore the information to obtain the second image data of the snapshot task.

[0025] In one of the embodiments, the backend processor includes a first storage server and a first business center;

[0026] The first storage server is used to receive the subscription information sent by the first business center, and send the subscription information to the camera end; it is also used to receive the first image data of the snapshot task sent by the camera end, and send the received first image data of the snapshot task to the first business center;

[0027] The first business center is used to generate the subscription information according to the subscription event set by the user, and send the subscription information to the first storage server; the first business center is also used to store the first image data received for the snapshot task.

[0028] In one embodiment: the first storage server is further used to receive the capture task event sent by the camera end, and send the capture task event to the first business center; in response to the index correspondence between the image to be captured of the capture task sent by the first business center and the video code stream of the first storage server, determine the video frame of the capture task, and send the video frame of the capture task to the first business center;

[0029] The first business center is also used to parse the received snapshot task event, obtain the index correspondence between the image to be captured of the snapshot task and the video code stream of the first storage server, and the OSD overlay information of the snapshot task, and send the index correspondence between the image to be captured of the snapshot task and the video code stream of the first storage server to the first storage server; the first business center is also used to encode the received video frame of the snapshot task to obtain the snapshot picture; based on the snapshot picture and the OSD overlay information of the snapshot task, obtain the second image data of the snapshot task, and store the obtained second image data of the snapshot task.

[0030] In one of the embodiments, the first storage server is also used to receive the capture task event sent by the camera end, and parse the received capture task event to obtain the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server; determine the video frame of the capture task based on the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server; encode the video frame of the capture task to obtain the captured picture; based on the captured picture and the OSD overlay information of the capture task, obtain the second image data of the capture task; and send the second image data of the capture task to the first business center.

[0031] In one of the embodiments, the backend processor includes a second storage server and a second business center;

[0032] The second business center is used to generate subscription information according to the subscription event set by the user, and send the subscription information to the second storage server; and is used to receive the first image data of the snapshot task sent by the camera end, and store the received first image data of the snapshot task;

[0033] The second business center is also used to receive the capture task event sent by the camera end, and parse the received capture task event to obtain the index correspondence between the image to be captured of the capture task and the video code stream of the second storage server, and the OSD overlay information of the capture task event; and send the index correspondence between the image to be captured and the video code stream of the second storage server to the second storage server; the second business center is also used to encode the received video frame of the capture task to obtain the captured picture; based on the captured picture and the OSD overlay information of the capture task, obtain the second image data of the capture task, and store the second image data of the capture task;

[0034] The second storage server is used to receive the information subscription information sent by the second business center and send the subscription information to the camera end; in response to the index correspondence between the image to be captured sent by the second business center and the video code stream of the second storage server, determine the video frame of the capture task, and send the video frame of the capture task to the second business center.

[0035] In a third aspect, the present application also provides a camera event capture system. The system includes:

[0036] The capture module is used to generate a new capture task for the events that need to be captured during the analysis of the camera's monitoring screen images, insert the new capture task into a capture task queue, and use the camera to capture the capture tasks in the capture task queue, and send the capture tasks that cannot be inserted into the capture task queue to the back-end processor connected to the camera for capture; the length of the capture task queue is related to the hardware performance of the camera and the current network traffic.

[0037] In a fourth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the event capture method of the camera described in the first aspect is implemented.

[0038] The above-mentioned event capture method, device, system and computer equipment generate new capture tasks for the events that need to be captured during the process of analyzing the monitoring screen image of the camera, insert them into the capture queue, use the camera to capture the capture tasks in the capture queue, and send the tasks that cannot be inserted into the capture queue to the back-end processor for capture. It gives priority to the front-end camera for capturing, and transfers the capture tasks that cannot be inserted into the capture task queue to the back-end processor for capturing in time. The device disperses a large number of snapshot tasks generated in a short period of time, thereby solving the problem of lost images / lost events during the snapshot process due to the generation of a large number of snapshot tasks in a short period of time. In addition, since the length of the snapshot task queue is related to the hardware performance of the camera and the current network traffic, when the current network traffic cannot normally transmit the data captured by the front-end camera due to the length of the snapshot task queue being too long, the length of the snapshot task queue can be adjusted to enable the data captured by the front-end camera according to the length of the snapshot task queue to be transmitted normally, thereby solving the problem of network freeze or packet loss during the transmission of snapshot data in the prior art.

[0039] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 A hardware structure block diagram of a terminal of a camera event capture method provided in an embodiment of the present application;

[0042] Figure 2 A flowchart of a method for capturing an event using a camera according to an embodiment of the present application;

[0043] Figure 3 A flowchart of a method for capturing an event using a camera provided in a preferred embodiment of the present application;

[0044] Figure 4 A structural block diagram of a first event capturing device provided in an embodiment of the present application;

[0045] Figure 5 A structural block diagram of a second event capturing device provided in an embodiment of the present application;

[0046] Figure 6A structural block diagram of a third event capturing device provided in an embodiment of the present application;

[0047] Figure 7 This is a structural block diagram of an event capture system of a camera provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal of a camera, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the event capture method of the camera of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the event capture method of the camera in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0053] In this embodiment, a camera event capture method is provided. Figure 2 is a flow chart of the event capture method of the camera of this embodiment, such as Figure 2 As shown, the process includes the following steps:

[0054] Step S210, in the process of analyzing the camera's surveillance screen image, generates a new snapshot task for the event that needs to be captured, inserts the new snapshot task into the snapshot task queue, and uses the camera to capture the snapshot tasks in the snapshot task queue, and sends the snapshot tasks that cannot be inserted into the snapshot task queue to the back-end processor connected to the camera for capture; the length of the snapshot task queue is related to the hardware performance of the camera and the current network traffic.

[0055] In this step, the process of analyzing the monitoring screen image of the camera can be a process of analyzing the monitoring screen image by technical means under specific conditions. Specifically, it can be a process of analyzing the monitoring screen image or video clip of the camera within a certain period of time through the image analysis module of the camera. Among them, the specific conditions can be certain conditions that require event capture for security monitoring, or certain conditions that require event capture for traffic control, etc. The purpose of analyzing the monitoring screen image of the camera is to distinguish the target events that need to be captured, and to capture the target events that need to be captured. It records the occurrence of the target event through event capture for subsequent analysis and evidence collection. The new capture task can be to perform algorithm analysis on the monitoring screen image of the camera at a preset frame rate to identify whether there is a capture trigger condition for the target event in the monitoring screen image. When the capture trigger condition for the target event exists in the monitoring screen image, a capture task for the target event is generated, that is, a new capture task. The algorithm analysis can be an image recognition algorithm, which can be edge detection, feature matching, etc. It should be noted that the specific algorithm analysis method is not specifically limited in this embodiment, as long as the algorithm analysis can be used to identify whether there is a capture trigger condition for the target event in the monitoring screen. The capture trigger condition of the above-mentioned target event can be motion detection (such as when a moving object appears in the video stream), specific event detection (such as recognition of a face, recognition of a license plate or recognition of a specific object, etc.) or environmental change (such as when the illumination change reaches a threshold, the temperature change reaches a threshold or the humidity change reaches a threshold), etc. The above-mentioned camera includes at least an image analysis module, a task generation module and an encoding module. The above-mentioned image analysis module is used to analyze the monitoring screen image or video clip of the camera. The above-mentioned task generation module is used to generate a capture task based on the analysis results of the image analysis module on the monitoring screen image or video clip of the camera. The above-mentioned encoding module is used to encode the capture task in the capture task queue.

[0056] The above-mentioned snapshot task queue may be a queue for storing snapshot tasks that need to be encoded using the camera's hardware encoder (camera's encoding module). It should be noted that when encoding the snapshot tasks in the snapshot task queue using the camera's encoding module, it is necessary to encode each snapshot task in the snapshot task queue in sequence from front to back. The length of the above-mentioned task queue is associated with the camera's hardware performance and current network traffic. Specifically, the length of the above-mentioned snapshot task queue may be determined based on the camera's hardware performance, encoding parameters, time delay threshold, and network traffic. The hardware performance of the above-mentioned camera may include the number of encoders in the camera, encoder performance, and memory of the camera device. The above-mentioned encoder is a hardware encoder. Generally, there is one or two encoders in a camera. Moreover, the encoder in the camera may have performance differences due to the use of chips from different manufacturers and models.

[0057] The above encoding parameters may include the resolution of the YUV image to be encoded, the encoding quality level, and the amount of OSD (On-Screen Display) overlay information. Among them, OSD overlay information refers to information such as text, icons, and timestamps superimposed on the video or image. Given the encoding parameters, the encoding time of the encoder is usually fixed, that is, it is predictable. Based on this, an empirical table of "encoding parameters-encoding time" under the current hardware platform can be established in advance.

[0058] For example, a chip hardware encoder: the YUV image resolution is 1080P, there is no OSD overlay information, and the JPG (Joint Photographic Experts Group, an image file format) encoding takes about 40ms (it should be noted that the time consumption here is about 40ms, not only the input and output time of the hardware encoder itself, but also includes the time consumption of the accompanying processing flow, such as YUV, OSD memory preparation, data copying, etc.); an additional line of 40-character OSD text (length less than 32) is superimposed, which takes about 8ms more; an additional rectangular frame is superimposed with a line width of 2 (pixels) and a length and width less than or equal to 960×540, which takes about 10ms more; that is, 25 1080P pictures without OSD overlay can be encoded in 1 second; because the algorithm processing also takes time, that is, a YUV image is sent to the algorithm for processing to generate a result event, which typically takes 60ms; therefore, the capture task queue length should be less than or equal to If the algorithm processes 12 frames, the upper limit of the events that can be processed is 2 events per frame (24 events per second). However, the number of events generated by 1 frame in actual scenes often far exceeds this value. If the negotiated bandwidth of the network card is 100Mbps, the network bandwidth is 100Mbps, that is, 12.5MB / s; when the number of events and the number of encodings are not limited, JPG encoding can reach 25 pictures per second. If the JPG encoding quality level is 5, a 1080P picture will occupy about 0.5MB after encoding, and 25 pictures will be 25×0.5MB=12.5MB; while the 1080P code stream uses H.264 encoding, 25 frames, and the code stream is 1Kbps~8Kbps, taking the typical value of 0.5MB / s. It can be seen that the amount of picture data is much larger than the amount of code stream data, and the sum of the two items exceeds the network bandwidth. Therefore, it is necessary to reduce the capture task queue to reduce the amount of picture data.

[0059] The above time delay threshold can be the time interval between the event generation and the event detection and the issuance of the snapshot task. Generally, the time delay threshold is a relatively short time length, for example, the time delay threshold is less than or equal to 1 second. The above network traffic can be the network bandwidth occupancy or the amount of data transmitted in the network when transmitting the data generated by the snapshot task.

[0060] In addition, the above-mentioned use of the camera to capture the snapshot tasks in the snapshot task queue may be to capture the snapshot tasks in the snapshot task queue using the camera in a preset order. Specifically, the above-mentioned use of the camera to capture the snapshot tasks in the snapshot task queue may be that the camera starts the acquisition of images or video streams based on the snapshot task requirements, performs preprocessing on the acquired images or video streams, such as noise reduction, white balance, etc., and performs trigger condition detection on the preprocessed images or video streams in real time. When the trigger condition is detected, the capture is immediately started to obtain the captured image or video. Then, the camera encodes the captured image or video, and superimposes the OSD overlay information at the same time to obtain the first image data of the snapshot task. The above-mentioned first image data is image data including the captured image and OSD overlay information.

[0061] It should be noted that the above-mentioned snapshot task queue is an empty queue in the initial state or when no snapshot task is inserted. As the snapshot task is inserted, the snapshot task queue will be filled. As the snapshot task is executed, new vacancies will appear in the snapshot task queue. If a large number of snapshot tasks are generated in a short period of time, and the speed of completing the snapshot task using the camera's encoder is lower than the speed at which the snapshot task is generated, there will inevitably be snapshot tasks that cannot be inserted into the snapshot task queue. At this time, the snapshot tasks that cannot be inserted into the snapshot task queue will be sent to the back-end processor connected to the camera for snapshot.

[0062] The back-end processor connected to the camera can be a device or software system that processes the video stream data from the front-end camera in the video surveillance system and performs functions such as video analysis, target detection, event recognition, data storage, etc. The back-end processor can be an independent hardware device, a software system running on a server, or a part of an embedded system. Generally, one back-end processor corresponds to multiple front-end cameras.

[0063] In the above step S210, during the process of analyzing the surveillance screen image of the camera, a new capture task is generated for the event that needs to be captured, and the event is inserted into the capture queue. The camera is used to capture the capture task in the capture queue, and the task that cannot be inserted into the capture queue is sent to the back-end processor for capture. By inserting the new capture task into the capture task queue and sending the task that cannot be inserted into the capture task queue to the back-end processor for capture, the front-end camera is given priority to capture, and the capture task that the front-end camera cannot complete in time is promptly transferred to the back-end device for capture. The back-end device disperses a large number of capture tasks generated in a short period of time, thereby solving the problem of image loss / event loss during the capture process due to the generation of a large number of capture tasks in a short period of time. In addition, because the length of the capture task queue is related to the hardware performance of the camera and the current network traffic, when the current network traffic cannot normally transmit the data captured by the front-end camera due to the length of the capture task queue being too long, the length of the capture task queue can be adjusted to achieve normal transmission of the data captured by the front-end camera according to the length of the capture task queue, thereby solving the problem of network freeze or packet loss during the transmission of the capture data in the prior art.

[0064] In one embodiment, based on step S210, in the process of analyzing the surveillance screen image of the camera, a new capture task is generated by analyzing the event that needs to be captured, the new capture task is inserted into the capture task queue, and the capture task in the capture task queue is captured by the camera, and the capture task that cannot be inserted into the capture task queue is sent to the back-end processor connected to the camera for capture, including:

[0065] Step S212, performing algorithm analysis on the surveillance screen image of the camera according to a preset frame rate, identifying the target event in the surveillance screen image, and generating a capture task of the target event based on the identification result of the target event.

[0066] The above-mentioned monitoring screen image can be an image in YUV format. The above-mentioned preset frame rate can be specifically set according to the specific scene requirements. If the event capture is mainly used to capture scenes with more details or instantaneous events, the frame rate can be set to a larger value, for example, the preset frame rate can be set to 60FPS or higher; if the event capture is mainly used to capture events with a longer duration, the frame rate can be set to a smaller value, for example, the preset frame rate can be set to 10FPS or lower. The above-mentioned algorithm analysis can be specifically set according to the specific situation, and can be divided into multiple algorithm analyses such as target detection, motion detection, behavior recognition, face recognition, license plate recognition, and anomaly detection according to needs. Among them, the above-mentioned algorithm analysis of target detection can include YOLO (You Only Look Once, single inference target detection algorithm) algorithm, Faster R-CNN (Fast Region-based Convolutional Neural Network, deep learning algorithm for target detection) algorithm, etc. The above-mentioned algorithm analysis of motion detection can include background subtraction method, optical flow method, etc. The algorithm analysis of the above-mentioned behavior recognition may include spatiotemporal convolutional networks, 3D convolutional networks, etc. The algorithm analysis of the above-mentioned face recognition may include deep learning models, feature point matching, etc. The algorithm analysis of the above-mentioned license plate recognition may include template matching, character segmentation and recognition, etc. The algorithm analysis of the above-mentioned anomaly detection may include statistical-based methods, deep learning-based methods, etc.

[0067] Specifically, the above-mentioned algorithmic analysis of the camera's monitoring screen image at a preset frame rate is performed to identify the target event in the monitoring screen image, and based on the identification result of the target event, a capture task of the target event is generated. It can be to use a preset method to perform algorithmic analysis on the camera's monitoring screen image at a preset frame rate to analyze whether there are capture trigger conditions for the target event in the monitoring screen image. When the capture trigger conditions for the target event exist in the monitoring screen image, a capture task of the target event is generated.

[0068] Step S214, insert the capture task of the target event into the capture task queue according to the priority of the preset event type, and use the camera to capture the capture task in the capture task queue to obtain the first image data of the capture task inserted into the capture task queue, convert the capture task that cannot be inserted into the capture task queue into a capture task event, and send it to the back-end processor connected to the camera for capture, to obtain the second image data of the capture task that cannot be inserted into the capture task queue; the capture task event includes the index correspondence between the image to be captured of the capture task and the video code stream of the storage server, and the OSD overlay information of the capture task.

[0069] In actual applications, users have different sensitivities to different events. Some need to be perceived and dealt with immediately (such as dangerous situations, accidents, etc.), some can delay dealing with them after perception (such as violations, etc.), and some only give prompts and archive them (such as status changes, etc.). That is, events can be classified by priority according to the severity of the event type, and high priority events are given priority. Among them, the priority of the above-mentioned preset event type can be the priority of the event type pre-set according to actual needs or actual application scenarios, and specifically can be the priority set by the user for each business type according to business needs. For example, event types can be divided into three priorities: high, medium and low. When the user sets a priority for each event type, the capture tasks can be sorted according to the priority of the corresponding business type, and the capture tasks of the business type with the highest priority can be given priority as needed.

[0070] Further, the capture task of the target event can be inserted into the capture task queue according to the priority of the preset event type. Specifically, the capture task of the target event can be inserted into the capture task queue according to the priority of the preset event type and the rule of the priority of the capture tasks in the capture task queue from high to low. When the capture task to be inserted has the same priority as the capture task in the capture task queue, the capture task to be inserted is inserted behind the capture task with the same priority.

[0071] The above-mentioned capture task event may be a non-image event with an index correspondence between the image when the capture task event is triggered and the video code stream of the storage server. The above-mentioned storage server may be a device or system specifically used to store video data and other related information from the camera in the video surveillance system. The above-mentioned storage server may be an independent hardware device, or a software system running on a general server, or even a cloud computing-based service. It should be noted that when using the storage server to store the video data of the camera, it is necessary to establish an index correspondence between the image to be captured of the capture task and the video code stream of the storage server in advance. The above-mentioned storage server can pull the stream after connecting to the camera, and at the same time, the camera can obtain the information of the storage server connected to the camera, such as the address (IP) of the storage server, etc. The above-mentioned index correspondence may include the storage server address, the correspondence between the camera video channel and the storage server recording channel, the timestamp, the frame number, the event type, the event priority, and the like. Through the above-mentioned index correspondence, the specific position of the image to be captured of the capture task in the video code stream of the storage server can be quickly located, so as to accurately capture the image to be captured.

[0072] The above-mentioned conversion of the snapshot task that cannot be inserted into the snapshot task queue into a snapshot task event, sending it to the back-end processor connected to the camera for snapshot, and obtaining the second image data of the snapshot task that cannot be inserted into the snapshot task queue, can be to convert the snapshot task that cannot be inserted into the snapshot task queue into a snapshot task event, and send it to the back-end processor connected to the camera, the back-end processor parses the received snapshot task event based on the received snapshot task event, obtains the index correspondence between the image to be captured of the snapshot task and the video code stream of the storage server, and the OSD overlay information of the snapshot task, and then determines the video frame of the snapshot task based on the index correspondence between the image to be captured of the snapshot task and the video code stream of the storage server, and then encodes the video frame of the snapshot task to obtain the snapshot picture, and obtains the second image data of the snapshot task based on the snapshot picture and the OSD overlay information of the snapshot task. The above-mentioned index correspondence between the image to be captured based on the capture task and the video code stream of the storage server is used to determine the video frame of the capture task. The storage server address, the correspondence between the camera video channel and the storage server recording channel, the timestamp, the frame number, and the event type are determined according to the index correspondence between the image to be captured based on the capture task and the video code stream of the storage server. Then, the video frame of the capture task is determined according to the storage server address, the correspondence between the camera video channel and the storage server recording channel, the timestamp, the frame number, and the event type.

[0073] Specifically, a camera may have multiple video channels (such as multiple sensors, that is, multiple lenses; the common one is only 1); the storage server has access restrictions (such as 64 channels, 128 channels, etc., each channel has a channel number), corresponding to the number of video channels that can be accessed by the camera. When the storage server accesses the camera, it will perform protocol interaction to obtain the basic information of the other party. At this time, the camera can know the IP address and access channel of the storage server; the camera itself can of course know that the algorithm is analyzing the video of its own channel at this time, corresponding to the information of the storage server, and obtain the channel correspondence. The video output by the camera is the original raw image output by the sensor. After ISP (image) processing, the YUV image is obtained, and then the H264 / 265 video stream is obtained through the encoder. Among them, YUV will be separated into one channel for the algorithm to process, that is, the camera end can know the correspondence between YUV and the (I, P) frame of the H264 / 265 video stream, assisted by information such as timestamp and frame number for easy indexing. The event type is convenient for association with the business protocol, so as to facilitate the construction of the event message body on the server side.

[0074] In the above steps S212 to S214, by performing algorithm analysis on the monitoring screen image of the camera, the target event in the monitoring screen image is identified, and the capture task of the target event is generated, and the newly generated capture task is inserted into the capture task queue according to the priority of the preset event type, and the capture task in the capture task queue is captured by the camera, and the capture task that cannot be inserted into the capture task queue is sent to the back-end processor for capture, which gives priority to the front-end camera for capture, and the capture task that the front-end camera cannot complete the capture in time is transferred to the back-end device for capture in time, and the large number of capture tasks generated in a short time are dispersed by the back-end device, which solves the problem of image loss / event loss in the capture process due to the large number of capture tasks generated in a short time. In addition, because the length of the capture task queue is associated with the hardware performance of the camera and the current network traffic, therefore, when the current network traffic cannot normally transmit the data captured by the front-end camera because the length of the capture task queue is too long, the length of the capture task queue can be adjusted to achieve the normal transmission of the data captured by the front-end camera according to the length of the capture task queue, which solves the problem of network jamming or packet loss in the transmission process of the capture data in the prior art.

[0075] Further, in one embodiment, after inserting the capture task of the target event into the capture task queue according to the priority of the preset event type, and using the camera to capture the capture task in the capture task queue, and obtaining the first image data of the capture task inserted into the capture task queue, the following steps are included:

[0076] Step S216: Based on the preset priority of the business data, the first image data is transmitted to a backend processor connected to the camera for storage.

[0077] When data is transmitted over the network, if it is not controlled, when the amount of data is large, it will cause network delay and packet loss, which is mainly manifested as: audio and video freeze, unresponsive device operation, etc. Generally, business data with a large amount of data has a great impact on network bandwidth, such as surveillance video, event pictures, etc. Therefore, the priority of the business data transmitted over the network can be set in advance, and the data with higher priority can be transmitted first by setting the priority of the business data. The priority can be sorted first according to the type of data transmitted over the network. For example, the priority of the preset business data can be that the code stream priority is higher than the device control, the priority of the device control is higher than the business configuration, and the priority of the business configuration is higher than the event. Among them, the event can include an event with a picture (the first data with a picture in this embodiment), an event with a capture task (the capture task event in this embodiment), and an event without a capture task and no picture (ordinary event). The above events can also be divided into important emergency events, important non-urgent events, urgent non-important events, and non-urgent non-important events according to the event type. The priority of event transmission can be set in the order of important emergency events, important non-urgent events, urgent non-important events, and non-urgent non-important events from high to low.

[0078] The above-mentioned method of transmitting the first image data to the back-end processor connected to the camera for storage based on the preset priority of the business data may be to transmit the first image data to the back-end processor connected to the camera for storage in the order of important and urgent events > important and non-urgent events > urgent and non-important events > non-urgent and non-important events based on the preset priority of the business data.

[0079] This step transmits the first image data to the back-end processor connected to the camera for storage based on the preset business data priority. It can give priority to the transmission of important data to ensure continuous flow and no frame loss during the transmission process. In addition, it can ensure that important emergency events are transmitted to the back-end processor with priority, so that important emergency events can be perceived and handled in time.

[0080] Specifically, in one embodiment, step S214 inserts the capture task of the target event into the capture task queue according to the priority of the preset event type, and uses the camera to capture the capture task in the capture task queue to obtain the first image data of the capture task inserted into the capture task queue, converts the capture task that cannot be inserted into the capture task queue into a capture task event, and sends it to the back-end processor connected to the camera for capture, and obtains the second image data of the capture task that cannot be inserted into the capture task queue, including:

[0081] Step S2142, when the snapshot task queue is idle, based on the preset event type priority, the snapshot task of the target event is inserted into the snapshot task queue, and the snapshot task in the snapshot task queue is captured by the camera to obtain the first image data.

[0082] Step S2144, when the snapshot task queue is full and the event type priority of the target event is higher than the event type priority corresponding to the snapshot task at the end of the snapshot task queue, based on the preset event type priority, the snapshot task of the target event is inserted into the snapshot task queue, and the snapshot tasks in the snapshot task queue are captured by the camera to obtain the first data with image, and the snapshot tasks squeezed out of the snapshot task queue are converted into snapshot task events, which are sent to the back-end processor for capturing to obtain the second data with image.

[0083] It should be noted that when the snapshot task queue is not idle and the priority of the event type of the target event is higher than the priority of the event type corresponding to the snapshot task at the end of the snapshot task queue, at this time, because the priority of the event type of the target event is higher than the priority of the event type corresponding to the snapshot task at the end of the snapshot task queue, the snapshot task of the target event can be inserted into the snapshot task queue. At the same time, the snapshot task originally at the end of the snapshot task queue will be squeezed out of the snapshot task queue, and the snapshot task squeezed out of the snapshot task queue will be converted into a snapshot task event, and the snapshot task event will be sent to the back-end processor for snapshot to obtain the second image data.

[0084] This step inserts the snapshot tasks into the non-idle snapshot task queue according to the priority of the preset event type, and sends the snapshot tasks squeezed out of the snapshot task queue to the back-end processor for snapshot. It gives priority to ensuring that the front-end camera captures important events, and transfers the snapshot tasks that the front-end camera cannot complete in time to the back-end device for snapshot in time. The back-end device disperses the unimportant tasks in the large number of snapshot tasks generated in a short period of time, thus solving the problem of lost images / events in the snapshot process due to the generation of a large number of snapshot tasks in a short period of time.

[0085] Step S2146, when the snapshot task queue is full and the event type priority of the target event is lower than or equal to the priority of the event type corresponding to the snapshot task at the end of the snapshot task queue, the snapshot task of the target event is converted into a snapshot task event and sent to the back-end processor for snapshot to obtain the second image data.

[0086] The above steps S2142 to S2146, by inserting the snapshot task of the target event into the snapshot task queue according to the priority of the preset event type, and using the camera to snapshot the snapshot task in the snapshot task queue, obtain the first image data of the snapshot task inserted into the snapshot task queue, convert the snapshot task that cannot be inserted into the snapshot task queue into a snapshot task event, and send it to the back-end processor connected to the camera for snapshot, to obtain the second image data of the snapshot task that cannot be inserted into the snapshot task queue, give priority to the front-end camera for snapshot, and transfer the snapshot task that the front-end camera cannot complete in time to the back-end device for snapshot in time, and disperse the large number of snapshot tasks generated in a short time by the back-end device, thereby solving the problem of lost images / lost events in the snapshot process due to the large number of snapshot tasks generated in a short time.

[0087] In one embodiment, the event capture method of the camera further includes:

[0088] Step S218, adjusting the length of the snapshot task queue based on the network traffic monitoring result.

[0089] The above network traffic monitoring results may include the amount of data transmitted in the network, the network bandwidth and the usage of the bandwidth, the uplink traffic and the downlink traffic, the traffic type, etc. Because the capture task queue is a queue of capture tasks encoded by the camera's hardware encoder, the longer the capture task queue is, the more tasks are encoded by the camera's hardware encoder, and the more first band image data is generated in a short time. When the capture task queue is shorter, the fewer tasks are encoded by the camera's hardware encoder, and the less first band image data is generated in a short time. Therefore, the length of the capture task queue can be adjusted based on the network traffic monitoring results to feedback and adjust the network traffic so that the network traffic can normally transmit the first band image data captured by the camera. Specifically, the current available bandwidth can be determined based on the network traffic monitoring results, and then according to the current available bandwidth and the amount of data in the capture task queue, it is judged whether the current available bandwidth can normally transmit the amount of data to be sent. If the current available bandwidth cannot normally transmit the amount of data to be sent, the amount of data to be sent is reduced by shortening the length of the capture task queue so that the available bandwidth can normally transmit the amount of data to be sent. Among them, the granularity of shortening the length of the snapshot task queue each time can be specifically set according to the specific situation. It should be noted that the network bandwidth can be able to normally transmit the amount of data to be sent by shortening the length of the snapshot task queue multiple times. If the current available bandwidth can normally transmit the amount of data to be sent, and the available bandwidth is greater than the preset third bandwidth threshold for a preset time length, the length of the snapshot task queue can be restored to the length of the snapshot task queue in the initial state.

[0090] This step adjusts the length of the snapshot task queue through network traffic monitoring results. Through load balancing and flow control, it can effectively cut peaks and control traffic to avoid long-term bandwidth occupancy, ensuring that the data captured by the front-end camera according to the length of the snapshot task queue can be transmitted normally, solving the problem of network freeze or packet loss in the transmission process of snapshot data in the prior art.

[0091] Further, in one embodiment, step S218, adjusting the length of the snapshot task queue based on the network traffic monitoring result, includes:

[0092] Step S2182, determining the current available bandwidth based on the network bandwidth and the current uplink traffic;

[0093] The above is based on the network bandwidth and the current uplink traffic to determine the current available bandwidth M. The specific process is as follows:

[0094] M=Nn

[0095] Where N is the network bandwidth and n is the current uplink traffic.

[0096] Step S2184, when the current available bandwidth is continuously less than the preset first bandwidth threshold within the preset time interval, or the bandwidth margin of the snapshot task queue after the snapshot task traffic application is less than the preset second bandwidth threshold, and the amount of data to be sent in the snapshot task queue is greater than the preset data amount threshold, shorten the length of the snapshot task queue.

[0097] The above-mentioned preset time interval can be specifically set according to the specific situation. By setting the preset time interval, the adjustment frequency of the snapshot task queue can be determined. If it is necessary to adjust the length of the snapshot task queue multiple times in a short period of time, the preset time interval can be set to a shorter time length. If it takes a longer time to adjust the length of the snapshot task queue once, the preset time interval can be set to a longer time length. For example, the preset time interval can be set to 10 minutes. The above-mentioned preset first bandwidth threshold can be specifically set according to the specific situation, and this embodiment does not make specific limitations here.

[0098] In addition, the controlled network connection C i (For example, there are k network connections in total) Before sending data, you need to apply for traffic m from the current available bandwidth i , and get the bandwidth margin R, among which, the data transmission task with high priority applies for traffic first, sends data after the application, and returns the bandwidth application after the sending is completed. If the data sending is not completed, it will apply for traffic next time and continue to send data.

[0099] The process of calculating the bandwidth margin R is as follows:

[0100]

[0101] The remaining bandwidth R is greater than or equal to 0, and j represents that there are j network connections whose capture task application traffic is captured by the capture task queue, j≤k, and i represents the i-th network connection.

[0102] The above-mentioned preset second bandwidth threshold can be specifically set according to specific circumstances, and this embodiment does not make any specific limitation here.

[0103] The amount of data to be sent in the snapshot task queue may be the amount of data of the snapshot tasks in the snapshot task queue for which no traffic has been applied.

[0104] The calculation method of the amount of data M1 to be sent in the above snapshot task queue is:

[0105]

[0106] In the above steps S2182 to S2184, the length of the snapshot task queue is adjusted through network traffic monitoring results. Through load balancing and flow control, the peak can be effectively cut and the flow can be controlled to avoid occupying the bandwidth for a long time, thereby ensuring that the data captured by the front-end camera according to the length of the snapshot task queue can be transmitted normally, thereby solving the problem of network freeze or packet loss in the transmission process of the snapshot data in the prior art.

[0107] The present embodiment is described and illustrated below through preferred embodiments.

[0108] Figure 3 FIG. 1 is a flow chart of a method for capturing an event using a camera provided in a preferred embodiment of the present application. Figure 3 As shown, the event capture method of the camera includes the following steps:

[0109] Step S310, performing algorithm analysis on the surveillance screen image of the camera according to a preset frame rate, identifying the target event in the surveillance screen image, and generating a capture task of the target event based on the recognition result of the target event;

[0110] Step S320, inserting the capture task of the target event into the capture task queue according to the priority of the preset event type, and using the camera to capture the capture task in the capture task queue to obtain the first image data of the capture task inserted into the capture task queue;

[0111] Step S330, based on the preset priority of the business data, transmitting the first image data to a backend processor connected to the camera for storage;

[0112] Step S340, converting the snapshot task that cannot be inserted into the snapshot task queue into a snapshot task event, sending it to the back-end processor connected to the camera for snapshot, and obtaining the second image data of the snapshot task that cannot be inserted into the snapshot task queue.

[0113] In the above steps S310 to S340, by inserting the new snapshot task into the snapshot queue, the snapshot tasks in the snapshot queue are captured by the camera, and the tasks that cannot be inserted into the snapshot queue are sent to the back-end processor for snapshot. By inserting the new snapshot task into the snapshot task queue and sending the tasks that cannot be inserted into the snapshot task queue to the back-end processor for snapshot, priority is given to the front-end camera for snapshot, and the snapshot tasks that the front-end camera cannot complete in time are promptly transferred to the back-end device for snapshot. The large number of snapshot tasks generated in a short period of time are dispersed by the back-end device, which solves the problem of lost images / lost events in the snapshot process due to the large number of snapshot tasks generated in a short period of time. In addition, because the length of the snapshot task queue is related to the hardware performance of the camera and the current network traffic, therefore, when the current network traffic cannot normally transmit the data captured by the front-end camera due to the length of the snapshot task queue being too long, the length of the snapshot task queue can be adjusted to achieve normal transmission of the data captured by the front-end camera according to the length of the snapshot task queue, thereby solving the problem of network freeze or packet loss in the transmission process of snapshot data in the prior art.

[0114] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0115] Based on the same inventive concept, an event capture device is also provided in this embodiment, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0116] In one embodiment, Figure 4is a structural block diagram of a first event capture device provided in an embodiment of the present application, such as Figure 4 As shown, the event capture device includes a camera end 42 and a back-end processor 44;

[0117] The camera end 42 is used to determine the target event in response to the subscription information sent by the back-end processor 44; the camera end 42 is also used to adjust the length of the snapshot task queue based on the network traffic monitoring result; the camera end 42 is also used to perform algorithm analysis on the camera's monitoring screen image at a preset frame rate, identify the target event in the monitoring screen image, and generate a snapshot task for the target event based on the identification result of the target event; the snapshot task of the target event is preferentially inserted into the snapshot task queue according to the priority of the event type, and the snapshot task in the snapshot task queue is captured by the camera to obtain the first image data of the snapshot task that has been inserted into the snapshot task queue, and the snapshot task that cannot be inserted into the snapshot task queue is converted into a snapshot task event, and sent to the back-end processor 44 for snapshot, to obtain the second image data of the snapshot task that cannot be inserted into the snapshot task queue; the camera end 42 is also used to transmit the first image data to the back-end processor 44 for storage in the order of the business data priority from high to low based on the preset business data priority;

[0118] The back-end processor 44 is used to generate subscription information according to the subscription event set by the user, and send the subscription information to the camera end 42; receive the first image data of the snapshot task sent by the camera end 42, and store the received first image data; the back-end processor 44 is also used to receive the snapshot task event sent by the camera end 42, and based on the received snapshot task event, determine the video frame corresponding to the snapshot task stored in the storage server; based on the video frame corresponding to the snapshot task stored in the storage server, capture the snapshot task and restore the information to obtain the second image data of the snapshot task.

[0119] The above-mentioned event capture device inserts a new capture task into a capture queue, uses a camera to capture the capture tasks in the capture queue, and sends the tasks that cannot be inserted into the capture queue to the back-end processor 44 for capture. It gives priority to the front-end camera for capturing, and transfers the capture tasks that the front-end camera cannot complete in time to the back-end device for capturing in time. The back-end device disperses a large number of capture tasks generated in a short period of time, thereby solving the problem of lost images / lost events in the capture process due to the generation of a large number of capture tasks in a short period of time. In addition, because the length of the capture task queue is related to the hardware performance of the camera and the current network traffic, therefore, when the current network traffic cannot normally transmit the data captured by the front-end camera due to the length of the capture task queue being too long, the length of the capture task queue can be adjusted to achieve normal transmission of the data captured by the front-end camera according to the length of the capture task queue, thereby solving the problem of network freeze or packet loss in the transmission process of the capture data in the prior art.

[0120] Furthermore, in one embodiment, Figure 5 is a structural block diagram of a second event capturing device provided in an embodiment of the present application, such as Figure 5 As shown, the backend processor 44 includes a first storage server 442 and a first business center 444;

[0121] The first storage server 442 is used to receive the subscription information sent by the first business center 444 and send the subscription information to the camera end 42; it is also used to receive the first image data of the snapshot task sent by the camera end 42, and send the received first image data of the snapshot task to the first business center 444;

[0122] The first business center 444 is used to generate subscription information according to the subscription event set by the user, and send the subscription information to the first storage server 442; the first business center 444 is also used to store the first image data of the received snapshot task.

[0123] In addition, in one embodiment, the first storage server 442 is also used to receive the capture task event sent by the camera end 42, and send the capture task event to the first business center 444; in response to the index correspondence between the image to be captured of the capture task sent by the first business center 444 and the video code stream of the first storage server 442, determine the video frame of the capture task, and send the video frame of the capture task to the first business center 444; the first business center 444 is also used to parse the received capture task event, obtain the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server 442, and the OSD overlay information of the capture task, and send the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server 442 to the first storage server 442; the first business center 444 is also used to encode the received video frame of the capture task to obtain a captured picture; based on the captured picture and the OSD overlay information of the capture task, obtain the second image data of the capture task, and store the obtained second image data of the capture task.

[0124] In one embodiment, the first storage server 442 is also used to receive the capture task event sent by the camera end 42, and parse the received capture task event to obtain the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server 442, as well as the OSD overlay information of the capture task; based on the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server 442, determine the video frame of the capture task; encode the video frame of the capture task to obtain a captured picture; based on the captured picture and the OSD overlay information of the capture task, obtain the second image data of the capture task; and send the second image data of the capture task to the first business center 444.

[0125] Additionally, in one embodiment, Figure 6 is a structural block diagram of a third event capturing device provided in an embodiment of the present application, such as Figure 6 As shown, the backend processor 44 includes a second storage server 446 and a second business center 448;

[0126] The second business center 448 is used to generate subscription information according to the subscription event set by the user, and send the subscription information to the second storage server 446; it is used to receive the first image data of the snapshot task sent by the camera end 42, and store the received first image data of the snapshot task;

[0127] The second business center 448 is also used to receive the capture task event sent by the camera end 42, and parse the received capture task event to obtain the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server 442, as well as the OSD overlay information of the capture task event; and send the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server 442 to the second storage server 446; the second business center 448 is also used to encode the received video frame of the capture task to obtain the captured picture; based on the captured picture and the OSD overlay information of the capture task, obtain the second image data of the capture task, and store the second image data of the capture task;

[0128] The second storage server 446 is used to receive the information subscription information sent by the second business center 448, and send the subscription information to the camera end 42; in response to the index correspondence between the image to be captured of the capture task sent by the second business center 448 and the video code stream of the first storage server 442, determine the video frame of the capture task, and send the video frame of the capture task to the second business center 448.

[0129] Based on the same inventive concept, a camera event capture system is also provided in this embodiment. The new system is used to implement the above embodiments and preferred implementation modes, and those that have been explained will not be repeated here.

[0130] In one embodiment, Figure 7 is a structural block diagram of an event capture system of a camera provided in an embodiment of the present application, such as Figure 7 As shown, the event capture system of the camera includes:

[0131] The capture module 72 is used to generate a new capture task for the events that need to be captured during the analysis of the camera's monitoring screen images, insert the new capture task into the capture task queue, and use the camera to capture the capture tasks in the capture task queue, and send the capture tasks that cannot be inserted into the capture task queue to the back-end processor 44 connected to the camera for capture; the length of the capture task queue is related to the hardware performance of the camera and the current network traffic.

[0132] The event capture system of the above-mentioned camera generates a new capture task for the event that needs to be captured during the analysis of the camera's monitoring screen image, and inserts the new capture task into the capture queue, uses the camera to capture the capture tasks in the capture queue, and sends the tasks that cannot be inserted into the capture queue to the back-end processor 44 for capture. It gives priority to the front-end camera for capturing, and transfers the capture tasks that the front-end camera cannot complete in time to the back-end device for capturing in time. The back-end equipment disperses the large number of snapshot tasks generated in a short period of time, thereby solving the problem of lost images / lost events during the snapshot process due to the large number of snapshot tasks generated in a short period of time. In addition, because the length of the snapshot task queue is related to the hardware performance of the camera and the current network traffic, when the current network traffic cannot normally transmit the data captured by the front-end camera due to the long length of the snapshot task queue, the length of the snapshot task queue can be adjusted to enable the data captured by the front-end camera according to the length of the snapshot task queue to be transmitted normally, thereby solving the problem of network freeze or packet loss during the transmission of snapshot data in the prior art.

[0133] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0134] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, any one of the camera event capture methods in the above embodiments is implemented.

[0135] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0136] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A camera event capture method, characterized in that: The method comprises: In the process of analyzing the camera's surveillance screen images, a new snapshot task is generated based on the events that need to be captured, the new snapshot task is inserted into a snapshot task queue, and the camera is used to capture the snapshot tasks in the snapshot task queue. The snapshot tasks that cannot be inserted into the snapshot task queue are sent to a back-end processor connected to the camera for capture; the length of the snapshot task queue is related to the hardware performance of the camera and the current network traffic.

2. The camera event capture method according to claim 1, characterized in that: In the process of analyzing the monitoring screen image of the camera, a new capture task is generated by analyzing the event that needs to be captured, the new capture task is inserted into the capture task queue, and the capture task in the capture task queue is captured by the camera, and the capture task that cannot be inserted into the capture task queue is sent to the back-end processor connected to the camera for capture, including: Performing algorithm analysis on the surveillance screen image of the camera at a preset frame rate, identifying the target event in the surveillance screen image, and generating a capture task of the target event based on the identification result of the target event; The capture task of the target event is inserted into the capture task queue according to the priority of the preset event type, and the camera is used to capture the capture task in the capture task queue to obtain the first image data of the capture task inserted into the capture task queue, and the capture task that cannot be inserted into the capture task queue is converted into a capture task event, and sent to the back-end processor connected to the camera for capture, to obtain the second image data of the capture task that cannot be inserted into the capture task queue; the capture task event includes the index correspondence between the image to be captured of the capture task and the video code stream of the storage server, and the OSD overlay information of the capture task.

3. The event capture method of the camera according to claim 2, characterized in that: After inserting the capture task of the target event into the capture task queue according to the priority of the preset event type, and using the camera to capture the capture task in the capture task queue, and obtaining the first image data of the capture task inserted into the capture task queue, the method includes: Based on the preset priority of the business data, the first image data is transmitted to the back-end processor connected to the camera for storage.

4. The camera event capture method according to claim 2, characterized in that: The method includes inserting the capture task of the target event into a capture task queue according to a preset event type priority, and using the camera to capture the capture task in the capture task queue to obtain the first image data of the capture task inserted into the capture task queue, converting the capture task that cannot be inserted into the capture task queue into a capture task event, and sending it to the back-end processor connected to the camera for capture, to obtain the second image data of the capture task that cannot be inserted into the capture task queue, including: When the snapshot task queue is idle, based on the preset event type priority, inserting the snapshot task of the target event into the snapshot task queue, and using the camera to capture the snapshot tasks in the snapshot task queue to obtain the first image data; When the snapshot task queue is not idle and the priority of the event type of the target event is higher than the priority of the event type corresponding to the snapshot task at the end of the snapshot task queue, based on the preset event type priority, the snapshot task of the target event is inserted into the snapshot task queue, and the snapshot tasks in the snapshot task queue are captured by the camera to obtain the first data with images, and the snapshot tasks squeezed out of the snapshot task queue are converted into the snapshot task events, which are sent to the back-end processor for capturing to obtain the second data with images; When the snapshot task queue is full and the priority of the event type of the target event is lower than or equal to the priority of the event type corresponding to the snapshot task at the end of the snapshot task queue, the snapshot task of the target event is converted into the snapshot task event, sent to the back-end processor for snapshot, and the second image data is obtained.

5. The event capture method of a camera according to any one of claims 3 or 4, characterized in that: Also includes: Based on the network traffic monitoring result, the length of the snapshot task queue is adjusted.

6. The event capture method of a camera according to claim 5, characterized in that: The adjusting the length of the snapshot task queue based on the network traffic monitoring result includes: Determine the current available bandwidth based on the network bandwidth and the current upstream traffic; When the current available bandwidth is continuously less than the preset first bandwidth threshold within a preset time interval, or the bandwidth margin of the snapshot task queue after the snapshot task traffic application is less than the preset second bandwidth threshold, and the amount of data to be sent in the snapshot task queue is greater than the preset data amount threshold, the length of the snapshot task queue is shortened.

7. An event capture device, characterized in that: The device includes a camera end and a back-end processor; The camera end is used to determine the target event in response to the subscription information sent by the back-end processor; the camera end is also used to adjust the length of the capture task queue based on the network traffic monitoring result; the camera end is also used to perform algorithm analysis on the camera's monitoring screen image according to a preset frame rate, identify the target event in the monitoring screen image, and generate a capture task for the target event based on the recognition result of the target event; The capture task of the target event is preferentially inserted into the capture task queue according to the priority of the event type, and the capture task in the capture task queue is captured by using a camera to obtain the first image data of the capture task that has been inserted into the capture task queue, and the capture task that cannot be inserted into the capture task queue is converted into a capture task event, and sent to the back-end processor for capture, to obtain the second image data of the capture task that cannot be inserted into the capture task queue; the camera end is also used to transmit the first image data to the back-end processor for storage in the order of the priority of the business data from high to low based on the priority of the preset business data; The backend processor is used to generate the subscription information according to the subscription event set by the user, and send the subscription information to the camera end; receive the first image data of the snapshot task sent by the camera end, and store the received first image data; The back-end processor is further used to receive the capture task event sent by the camera end, and determine the video frame corresponding to the capture task stored in the storage server based on the received capture task event; Based on the video frames corresponding to the snapshot task stored in the storage server, the snapshot task is captured and information is restored to obtain second image data of the snapshot task.

8. The event capture device according to claim 7, characterized in that: The backend processor includes a first storage server and a first business center; The first storage server is used to receive the subscription information sent by the first business center, and send the subscription information to the camera end; it is also used to receive the first image data of the snapshot task sent by the camera end, and send the received first image data of the snapshot task to the first business center; The first business center is used to generate the subscription information according to the subscription event set by the user, and send the subscription information to the first storage server; the first business center is also used to store the first image data received for the snapshot task.

9. The event capture device according to claim 8, characterized in that: The first storage server is further configured to receive the capture task event sent by the camera end, and send the capture task event to the first business center; in response to the index correspondence between the image to be captured of the capture task sent by the first business center and the video code stream of the first storage server, determine the video frame of the capture task, and send the video frame of the capture task to the first business center; The first business center is also used to parse the received snapshot task event, obtain the index correspondence between the image to be captured of the snapshot task and the video code stream of the first storage server, and the OSD overlay information of the snapshot task, and send the index correspondence between the image to be captured of the snapshot task and the video code stream of the first storage server to the first storage server; the first business center is also used to encode the received video frame of the snapshot task to obtain the snapshot picture; based on the snapshot picture and the OSD overlay information of the snapshot task, obtain the second image data of the snapshot task, and store the obtained second image data of the snapshot task.

10. The event capture device according to claim 8, characterized in that: The first storage server is further used to receive the capture task event sent by the camera end, and parse the received capture task event to obtain the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server, as well as the OSD overlay information of the capture task; Based on the index correspondence between the image to be captured of the capture task and the video code stream of the first storage server, the video frame of the capture task is determined; the video frame of the capture task is encoded to obtain the captured image; based on the captured image and the OSD overlay information of the capture task, the second image data of the capture task is obtained; and the second image data of the capture task is sent to the first business center.

11. The event capture device according to claim 7, characterized in that: The backend processor includes a second storage server and a second business center; The second business center is used to generate subscription information according to the subscription event set by the user, and send the subscription information to the second storage server; and is used to receive the first image data of the snapshot task sent by the camera end, and store the received first image data of the snapshot task; The second business center is further used to receive the capture task event sent by the camera end, and parse the received capture task event to obtain the index correspondence between the image to be captured of the capture task and the video stream of the second storage server, as well as the OSD overlay information of the capture task event; and sending the index correspondence between the image to be captured and the video code stream of the second storage server to the second storage server; the second business center is also used to encode the received video frame of the capture task to obtain the captured image; based on the captured image and the OSD overlay information of the capture task, obtain the second image data of the capture task, and store the second image data of the capture task; The second storage server is used to receive the information subscription information sent by the second business center and send the subscription information to the camera end; in response to the index correspondence between the image to be captured sent by the second business center and the video code stream of the second storage server, determine the video frame of the capture task, and send the video frame of the capture task to the second business center.

12. A camera event capture system, characterized in that: The system comprises: The capture module is used to generate a new capture task for the events that need to be captured during the analysis of the camera's monitoring screen images, insert the new capture task into a capture task queue, and use the camera to capture the capture tasks in the capture task queue, and send the capture tasks that cannot be inserted into the capture task queue to the back-end processor connected to the camera for capture; the length of the capture task queue is related to the hardware performance of the camera and the current network traffic.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the event capture method of the camera according to any one of claims 1 to 6 are implemented.

Images