All-round video processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a surround-view video processing method and device, computer equipment, a computer readable storage medium and a computer program product, and relates to the technical field of computer vision. The method comprises the following steps: acquiring an all-round video file obtained by splicing a plurality of original video files; the plurality of original video files are acquired in real time by a plurality of cameras at preset angles and positions; according to at least one preset target event identification condition, identifying the all-round video file in parallel, and according to an identification result corresponding to each target event identification condition, performing video extraction on the all-round video file to obtain a target video clip corresponding to each target event identification condition; classifying the target video clip by adopting a pre-trained classification model to obtain a classification result; and according to the abnormal event included in the classification result and the target video clip corresponding to the abnormal event, constructing an association table and storing the association table. By adopting the method, the video backtracking efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a surround view video processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] 360-degree surround video recording is set up in a specific scene. There are two main ways to record 360-degree surround video. One is the pan-tilt mode, which uses timed polling or manual polling, and the other is a fixed-angle gun camera. After installation, the position is fixed and cannot be changed. If you need to cover a certain position, you need a combination of multiple cameras. Usually, the acquired surveillance video needs to be reviewed and path tracked. At present, because surround surveillance is shot by multiple cameras, the video backtracking and path tracking methods need to be reviewed for each camera, and finding a specific location is time-consuming and labor-intensive. Therefore, the current surround video processing method has the problem of low video backtracking efficiency. Summary of the invention

[0003] Based on this, it is necessary to provide a surround view video processing method, device, computer equipment, computer readable storage medium and computer program product that can improve the efficiency of video retracing in response to the above technical problems.

[0004] In a first aspect, the present application provides a surround view video processing method, comprising:

[0005] Acquire a surround view video file obtained by stitching together multiple original video files; the multiple original video files are acquired in real time by multiple cameras at preset angles and positions;

[0006] According to at least one preset target event recognition condition, the surround view video file is recognized in parallel, and video extraction is performed on the surround view video file according to the recognition results corresponding to each of the target event recognition conditions to obtain a target video segment corresponding to each of the target event recognition conditions;

[0007] Using a pre-trained classification model to classify the target video clip to obtain a classification result; the classification result includes abnormal events and normal events;

[0008] According to the abnormal events included in the classification results and the target video segments corresponding to the abnormal events, a correlation table is constructed and stored.

[0009] In one embodiment, obtaining a surround view video file obtained by splicing multiple original video files includes:

[0010] Using an image stitching algorithm to stitch the original video files together to obtain an initial stitched video file;

[0011] A weighted average fusion algorithm is used to fuse the stitching seams of the initial stitching video file to obtain the surround view video file.

[0012] In one of the embodiments, the surround view video file is identified in parallel according to at least one preset target event identification condition, and video extraction is performed on the surround view video file according to the identification results corresponding to each of the target event identification conditions to obtain a target video segment corresponding to each of the target event identification conditions, including:

[0013] In the case of tracing the target event, the surround video file is split according to a preset time interval to obtain a plurality of video clips;

[0014] For each video clip, the video clip is recognized in parallel according to each of the target event recognition conditions, and recognition results of the video clip by each of the target event recognition conditions are obtained; wherein the recognition of each of the video clips is performed in parallel;

[0015] The recognition results of the video clips by the target event recognition conditions are used as the recognition results corresponding to the target event recognition conditions.

[0016] In one embodiment, the video extraction is performed on the surround view video file according to the recognition results corresponding to each of the target event recognition conditions to obtain the target video segment corresponding to each of the target event recognition conditions, including:

[0017] Identify the surround video file according to the morphological structure elements corresponding to the target event to determine the moving target;

[0018] A tracking algorithm is used to track the behavior trajectory of the moving target, and a target video segment corresponding to the target event is determined from the surround video file according to the behavior trajectory of the moving target.

[0019] In one embodiment, after the target video clip is classified using a pre-trained classification model and a classification result is obtained, the method further includes:

[0020] When the classification result is that the target event is an abnormal event, analyzing the number of moving targets and the behavior trajectories corresponding to each abnormal event to generate a safety report;

[0021] Determine the alarm method corresponding to the abnormal event, and issue an alarm according to the alarm method.

[0022] In one of the embodiments, after constructing an association table based on the abnormal events included in the classification results and the target video segments corresponding to the abnormal events and storing the association table, the method further includes:

[0023] In response to a retrieval operation or a selection operation for key information of the abnormal event, a target video segment of the abnormal event corresponding to the key information is determined according to the association table, and the target video segment is displayed.

[0024] In a second aspect, the present application further provides a surround view video processing device, comprising:

[0025] A video acquisition module is used to acquire a surround video file obtained by splicing multiple original video files; the multiple original video files are acquired in real time by multiple cameras at preset angles and positions;

[0026] An event recognition module is used to recognize the surround view video file in parallel according to at least one preset target event recognition condition, and to extract the video from the surround view video file according to the recognition results corresponding to each target event recognition condition to obtain a target video segment corresponding to each target event recognition condition;

[0027] An event classification module, used to classify the target video clip using a pre-trained classification model to obtain a classification result; the classification result includes abnormal events and normal events;

[0028] The video storage module is used to construct an association table and store the association table according to the abnormal events included in the classification results and the target video clips corresponding to the abnormal events.

[0029] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0030] Acquire a surround view video file obtained by stitching together multiple original video files; the multiple original video files are acquired in real time by multiple cameras at preset angles and positions;

[0031] According to at least one preset target event recognition condition, the surround view video file is recognized in parallel, and video extraction is performed on the surround view video file according to the recognition results corresponding to each of the target event recognition conditions to obtain a target video segment corresponding to each of the target event recognition conditions;

[0032] Using a pre-trained classification model to classify the target video clip to obtain a classification result; the classification result includes abnormal events and normal events;

[0033] According to the abnormal events included in the classification results and the target video segments corresponding to the abnormal events, a correlation table is constructed and stored.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0035] Acquire a surround view video file obtained by stitching together multiple original video files; the multiple original video files are acquired in real time by multiple cameras at preset angles and positions;

[0036] According to at least one preset target event recognition condition, the surround view video file is recognized in parallel, and video extraction is performed on the surround view video file according to the recognition results corresponding to each of the target event recognition conditions to obtain a target video segment corresponding to each of the target event recognition conditions;

[0037] Using a pre-trained classification model to classify the target video clip to obtain a classification result; the classification result includes abnormal events and normal events;

[0038] According to the abnormal events included in the classification results and the target video segments corresponding to the abnormal events, a correlation table is constructed and stored.

[0039] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0040] Acquire a surround view video file obtained by stitching together multiple original video files; the multiple original video files are acquired in real time by multiple cameras at preset angles and positions;

[0041] According to at least one preset target event recognition condition, the surround view video file is recognized in parallel, and video extraction is performed on the surround view video file according to the recognition results corresponding to each of the target event recognition conditions to obtain a target video segment corresponding to each of the target event recognition conditions;

[0042] Using a pre-trained classification model to classify the target video clip to obtain a classification result; the classification result includes abnormal events and normal events;

[0043] According to the abnormal events included in the classification results and the target video segments corresponding to the abnormal events, a correlation table is constructed and stored.

[0044] The above-mentioned surround video processing method, device, computer equipment, computer-readable storage medium and computer program product, by obtaining a surround video file obtained by splicing multiple original video files, the multiple original video files are collected in real time by multiple cameras at preset angles and positions, and the panoramic monitoring of the monitoring area range is realized through the surround video file. And according to at least one preset target event recognition condition, the surround video file is recognized in parallel, and the surround video file is extracted according to the recognition results corresponding to each target event recognition condition, and the target video clip corresponding to each target event recognition condition is obtained. Through the surround video file, the event monitoring occurring in the monitoring range can be realized by identifying the target event. The target video clip is classified using a pre-trained classification model to obtain a classification result, wherein the classification result includes abnormal events and normal events. The recognition of abnormal events is realized, which facilitates timely taking corresponding measures for the monitoring area. According to the abnormal events included in the classification results and the target video clips corresponding to the abnormal events, an association table is constructed and stored, which facilitates backtracking and retrieval of abnormal events. In summary, by splicing the original video files collected by multiple cameras to obtain a surround view video file, monitoring and identification of abnormal events are performed based on the surround view video file, avoiding video identification and tracing for each camera in turn, and improving the efficiency of video backtracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A diagram showing an application environment of a surround view video processing method in an embodiment;

[0047] Figure 2 1 is a flow chart of a surround view video processing method in one embodiment;

[0048] Figure 3 A schematic diagram of a process of extracting a target video segment in one embodiment;

[0049] Figure 4 is a structural block diagram of a surround view video processing device in one embodiment;

[0050] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] The surround view video processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The application scenario includes: a server 102, a data storage system and multiple cameras 104. Among them, the server 102 communicates with the camera 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 102, or it can be placed on the cloud or other network servers. The server 102 can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides cloud computing services. The server 102 obtains a surround video file obtained by splicing multiple original video files, wherein the multiple original video files are obtained by real-time acquisition by multiple cameras 104 at preset angles and positions. The server 102 recognizes the surround video files in parallel according to at least one preset target event recognition condition, and extracts the surround video files according to the recognition results corresponding to each target event recognition condition, and obtains the target video clip corresponding to each target event recognition condition. The server 102 further uses a pre-trained classification model to classify the target video clip to obtain a classification result. Among them, the classification result includes abnormal events and normal events. The server 102 constructs an association table according to the abnormal events and the target video segments corresponding to the abnormal events included in the classification results, and stores the association table.

[0053] In an exemplary embodiment, Figure 2 As shown, a surround video processing method is provided, and the method is applied to Figure 1 The server 102 in the example is used as an example to illustrate the method, which includes the following steps S202 to S208. Among them:

[0054] Step S202, obtaining a surround view video file obtained by splicing multiple original video files.

[0055] The multiple original video files are collected in real time by multiple cameras at preset angles and positions. The layout of the multiple cameras should cover a 360-degree monitoring range to achieve a panoramic monitoring range, and the monitoring ranges of the cameras may overlap.

[0056] The surround video file may be a 360-degree panoramic video file, and the scene within a 360-degree range may be surrounded and viewed during video viewing.

[0057] Optionally, the server obtains multiple original video files captured by multiple cameras for the monitoring area, and obtains a surround video file based on the splicing of the multiple original video files, so as to view the entire monitoring range of the monitoring area through one video file.

[0058] Step S204, according to at least one preset target event recognition condition, the surround view video file is recognized in parallel, and video extraction is performed on the surround view video file according to the recognition results corresponding to each target event recognition condition to obtain a target video segment corresponding to each target event recognition condition.

[0059] The target event may be an object and its behavior that needs to be monitored in the monitoring area. The target event recognition condition may be the basis for identifying the monitored object and its behavior through the video. For example, if the target event is to monitor the vehicle behavior in the parking lot, the recognition condition is the preset vehicle profile.

[0060] The target video segment may be a video segment corresponding to a time period in which the target event occurs in the surround video file.

[0061] Optionally, the server recognizes the surround video in parallel according to at least one preset target recognition condition, if there are multiple target recognition conditions, and obtains multiple recognition results through multi-threaded recognition. According to the recognition results corresponding to each target recognition condition, the surround video file is extracted to obtain the target video clip corresponding to each target event recognition condition. For example, the target event recognition condition is human behavior, and the recognition result is that human behavior is recognized between time point A and time point B in the surround video file. The video between time point A and time point B in the surround video file is extracted to obtain the video clip corresponding to the target event recognition condition.

[0062] Step S206: classify the target video clip using the pre-trained classification model to obtain a classification result.

[0063] The pre-trained classification model is trained by video files and classification labels corresponding to each video.

[0064] The classification results include abnormal events and normal events, that is, the target event corresponding to the target video clip is an abnormal event or a normal event.

[0065] Optionally, the server uses a pre-trained classification model to classify the target video segment, obtains a classification result corresponding to the target video segment, and determines whether the target event corresponding to the target video segment is an abnormal event or a normal event.

[0066] Step S208: constructing and storing an association table according to the abnormal events and the target video clips corresponding to the abnormal events included in the classification results.

[0067] The association table may be a mapping table constructed between an identifier of an abnormal event and a target video clip, and the identifier may be relevant information of the abnormal event such as time, location, target, etc.

[0068] Optionally, the server constructs an association table based on the abnormal events and target video clips corresponding to the abnormal events included in the classification results, using key information of the abnormal events as identifiers, and stores the association table in a data storage system. The association table can also be compressed when stored to save data storage space.

[0069] In the above-mentioned surround video processing method, by obtaining a surround video file obtained by splicing multiple original video files, the multiple original video files are collected in real time by multiple cameras at preset angles and positions, and the panoramic monitoring of the monitoring area range is realized through the surround video file. And according to at least one preset target event recognition condition, the surround video file is recognized in parallel, and the surround video file is extracted according to the recognition results corresponding to each target event recognition condition, and the target video clip corresponding to each target event recognition condition is obtained. Through the surround video file, the event monitoring occurring in the monitoring range can be realized by identifying the target event. The target video clip is classified using a pre-trained classification model to obtain a classification result, wherein the classification result includes abnormal events and normal events. The recognition of abnormal events is realized, which facilitates timely taking corresponding measures for the monitoring area. According to the abnormal events included in the classification results and the target video clips corresponding to the abnormal events, an association table is constructed and stored, which facilitates backtracking and retrieval of abnormal events. In summary, by splicing the original video files collected by multiple cameras to obtain a surround view video file, monitoring and identification of abnormal events are performed based on the surround view video file, avoiding video identification and tracing for each camera in turn, and improving the efficiency of video backtracking.

[0070] In an exemplary embodiment, step S202 obtains a surround view video file obtained by splicing multiple original video files, including:

[0071] An image stitching algorithm is used to stitch the original video files to obtain an initial stitching video file; a weighted average fusion algorithm is used to fuse the stitching seams of the initial stitching video file to obtain a surround view video file.

[0072] Among them, the image stitching algorithm can be a technology for synthesizing multiple images into a panoramic image.

[0073] Among them, the weighted average fusion algorithm can be a common image fusion method, especially in image stitching, used to process the fusion of overlapping areas. In the overlapping area, the weighted average method is used to smoothly connect the two images. The basic idea is to perform weighted calculation according to the weights of the two images in the overlapping area. Among them, the stitching seam can be the overlapping area of ​​the two images.

[0074] Optionally, the server uses an image stitching algorithm to stitch the original video files to obtain an initial stitched video file. Specifically, the server uses a feature point monitoring algorithm to identify key points in the video image, and then uses feature descriptors to match key points to find similar areas between the original video files. The transformation matrix is ​​calculated based on the matched feature points, and the video is aligned to a common coordinate system to obtain an initial stitched video file. The server further uses a weighted average fusion algorithm to fuse the stitching seams of the initial stitching file, determines the weight of each pixel based on the pixel position, depth, brightness and other information of the video, and performs a weighted average calculation on each pixel of the stitching seam according to the weight to obtain the fused pixel value, and merges the fused result with the video of the non-overlapping area to obtain a surround video file.

[0075] In this embodiment, the original video files captured by multiple cameras are stitched and fused to obtain a surround video file, so that the monitoring range of multiple cameras can be monitored through the surround video file, avoiding the need to call and view the monitoring area of ​​each camera in turn, thereby improving the efficiency of video backtracking.

[0076] In an exemplary embodiment, step S204 recognizes the surround video file in parallel according to at least one preset target event recognition condition, extracts the surround video file according to the recognition results corresponding to each target event recognition condition, and obtains the target video segment corresponding to each target event recognition condition, including:

[0077] In the case of target event tracing, the surround video file is split according to preset time intervals to obtain multiple video clips; for each video clip, the video clip is identified in parallel according to each target event recognition condition to obtain recognition results of the video clip by each target event recognition condition; wherein, the recognition of each video clip is performed in parallel; the recognition results of the video clip by each target event recognition condition are used as the recognition results corresponding to each target event recognition condition.

[0078] The target event may be an event other than the event recognized by the server in real time. Since the event has not been recognized in real time, the surround video within the historical time period needs to be retroactively recognized for the event.

[0079] Optionally, when tracing the target event, the server splits the surround video file according to a preset time interval to obtain multiple video clips. The preset time interval is determined according to the maximum computing power of the server, such as how many videos the server can support for event recognition at the same time, and the preset time interval is determined according to the number of videos. For each video clip, the video clips are recognized in parallel according to the target event recognition conditions to obtain recognition results of the video clips for each target event recognition condition. Among them, the recognition of each video clip is performed in parallel. If there are multiple target events, the recognition of each target event is also performed in parallel. The recognized video clips included in the recognition results of each target event recognition condition for the video clip are spliced ​​according to the timeline as the recognition results corresponding to each target event recognition condition.

[0080] In this embodiment, when tracing a target event, the surround video file of the historical time period is split according to a preset time interval, and each video segment is recognized in parallel, which improves the resource utilization of the server and further improves the video tracing efficiency.

[0081] In an exemplary embodiment, Figure 3 As shown, step S204 extracts the surround video file according to the recognition results corresponding to each target event recognition condition to obtain the target video segment corresponding to each target event recognition condition, including the following steps S302 to S304. Among them:

[0082] Step S302: identifying the surround video file according to the morphological structure elements corresponding to the target event to determine the moving target.

[0083] The morphological structure elements may be key features of the morphological structure of the object corresponding to the target event, such as corner points, edges, etc.

[0084] Optionally, according to the monitoring object corresponding to the target event, a screenshot of the surround video file is taken, and the edge of the object is detected by an image edge detection algorithm to obtain a morphological structure element. The server searches for an area matching the morphological structure element for each frame in the surround video file according to the morphological structure element corresponding to the target event, thereby determining the moving target.

[0085] Step S304: Track the behavior trajectory of the moving target using a tracking algorithm, and determine the target video segment corresponding to the target event from the surround video file according to the behavior trajectory of the moving target.

[0086] Optionally, the server uses a tracking algorithm to obtain the behavior trajectory of the moving target based on the position information of the morphological structural elements that track the moving target in each frame in the surround video file. The position information can be obtained by positioning a spatial pixel coordinate network constructed with a designated point in an area monitored by multiple cameras as a coordinate far point. Further, based on the behavior trajectory of the moving target, a target video segment corresponding to the target time corresponding to the starting point and the end point of the behavior trajectory is determined from the surround video file.

[0087] In this embodiment, the moving target is identified from the surround video file through the morphological structure elements corresponding to the target event, and the trajectory of the moving target is determined by the tracking algorithm. The target video clip corresponding to the target event is obtained based on the trajectory of the moving target, which provides basic data support for subsequent behavior analysis and anomaly detection, and helps to identify potential safety hazards or abnormal behaviors.

[0088] In an exemplary embodiment, step S206 classifies the target video clip using a pre-trained classification model, and after obtaining the classification result, further includes:

[0089] When the classification result is that the target event is an abnormal event, an analysis is performed based on the number of moving targets and behavior trajectories corresponding to each abnormal event to generate a safety report; the alarm method corresponding to the abnormal event is determined, and an alarm is issued according to the alarm method.

[0090] Among them, the safety report can include relevant data of historical abnormal events, the causes and response strategies, which can serve as a basis or reference for subsequent monitoring optimization.

[0091] Optionally, when the classification result is that the target event is an abnormal event, a trend analysis is performed based on the number of moving targets corresponding to each abnormal event, and the number trend changes of moving targets in different time periods are analyzed; behavior analysis is performed based on the behavior trajectory of the moving targets, and the causes of abnormal events are analyzed, etc. And correlation analysis is performed on the correlation between each abnormal event, such as determining whether there is a causal relationship between certain abnormal events. A safety report is generated based on the above analysis results. The server further determines the alarm method corresponding to the abnormal event and issues an alarm according to the alarm method. For example, if the abnormal event is a traffic accident, an audible and visual alarm is issued to the monitoring staff; if the abnormal event is an abnormal operation of the monitored equipment, the alarm method is to shut down the power system of the equipment and issue a prompt, etc.

[0092] In this embodiment, by generating a safety report and an intelligent alarm mechanism through in-depth analysis of abnormal events, not only the monitoring and response efficiency of abnormal events is improved, but also a scientific basis and decision-making support are provided for safety management.

[0093] In an exemplary embodiment, after constructing and storing a correlation table according to the abnormal events and the target video segments corresponding to the abnormal events included in the classification results, the following further includes:

[0094] In response to a search operation or a selection operation for key information of an abnormal event, a target video segment of the abnormal event corresponding to the key information is determined according to the association table, and the target video segment is displayed.

[0095] The key information of the abnormal event may be an identifier set when constructing the association table, including information such as the time, location, and target of the abnormal event.

[0096] The search operation may be a typing operation on an input box component displayed by the system, and the association table may be searched through the typed information.

[0097] The selection operation may be a trigger operation for an identifier of an association table displayed by the system.

[0098] Optionally, in response to a retrieval operation or a selection operation for key information of an abnormal event, the server queries an associated table based on the key information entered in the retrieval operation or the key information selected in the selection operation, determines a target video clip of the abnormal event corresponding to the key information, displays the video clip through the system's display screen, and may also support downloading of the target video clip.

[0099] In this embodiment, the server can accurately query and display the target video clip corresponding to the abnormal event based on the key information of the abnormal event input or selected by the user, and realize the playback and export of the video of the abnormal event. This not only improves the efficiency and accuracy of information retrieval, but also provides users with intuitive visual support, enhancing decision-making ability and risk management.

[0100] In an exemplary embodiment, a video surround view tracing method is provided, comprising:

[0101] Step 1: Multi-camera layout, equipped with multiple high-definition cameras, covering a full range of monitoring. These cameras will be arranged at different angles and positions to achieve panoramic monitoring and ensure that there are no blind spots. Through multi-view data fusion, more comprehensive and accurate monitoring information is provided.

[0102] Step 2, video stream processing and analysis of the video captured by the camera, will use advanced video stream processing and analysis technology to process the video stream captured by the camera in real time. This includes functions such as video encoding, image enhancement, motion detection, target recognition and tracking to achieve intelligent monitoring and event detection. Using deep learning algorithms, accurate recognition and behavior analysis of targets such as people, vehicles, and objects can be achieved, abnormal events, violations of behavioral rules, and other security risks can be identified and marked, and alarms can be automatically triggered or relevant personnel can be notified.

[0103] Step 3: Use deep learning algorithms to automatically analyze and trace back the events identified in step 2. Based on key information such as the time, location, and target of a specific event, relevant videos at that specific time can be quickly retrieved and reproduced, providing faster and more accurate event tracing capabilities.

[0104] Step 4: Store the identified and analyzed videos with high-capacity storage devices for long-term storage of surveillance video data. At the same time, the product will also provide convenient data management functions such as video retrieval, playback and export to support users' manual verification of specific events or time periods. Through intelligent data compression and storage strategies, storage efficiency and capacity utilization are maximized.

[0105] Step 5: Provide early warning or remote operation for the identified abnormal events. Provide remote and local access and control functions, support integration with other security systems, such as alarm systems, access control systems, etc., to achieve comprehensive security management, and directly handle specific cases through linkage and self-processing: sound and light alarms, valves, lock control, etc.

[0106] Step 6: Conduct in-depth mining based on the analyzed data to extract valuable information and patterns. Based on these analysis results, the product can generate real-time security reports, behavioral trend analysis and predictions, and provide users with intelligent decision-making and optimized security strategies based on historical processing methods.

[0107] Step 7 supports downloading custom algorithm models from the cloud. When the device receives the relevant information, it starts to trace back the history and conduct relevant analysis based on the business, and packages the analysis results and transmits them back to achieve rapid realization of data assets and apply data assets to other related fields.

[0108] In this embodiment, the functions of all-round monitoring coverage, intelligent event detection and automatic backtracking analysis will be realized. Through the combination of multi-camera layout, video stream processing and analysis, artificial intelligence backtracking technology, data storage and management, access and control, data analysis and intelligent decision-making, the product will provide users with efficient and accurate security monitoring and event tracing capabilities, effectively respond to various security risks and threats, and improve the level of security management.

[0109] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0110] Based on the same inventive concept, the embodiment of the present application also provides a surround view video processing device for implementing the surround view video processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more surround view video processing device embodiments provided below can refer to the limitations of the surround view video processing method above, and will not be repeated here.

[0111] In an exemplary embodiment, Figure 4 As shown, a surround view video processing device 400 is provided, comprising: a video acquisition module 402, an event recognition module 404, an event classification module 406 and a video storage module 408, wherein:

[0112] The video acquisition module 402 is used to acquire a surround video file based on the splicing of multiple original video files; the multiple original video files are acquired in real time by multiple cameras at preset angles and positions.

[0113] The event recognition module 404 is used to recognize the surround video files in parallel according to at least one preset target event recognition condition, and to extract the video from the surround video files according to the recognition results corresponding to each target event recognition condition to obtain the target video segment corresponding to each target event recognition condition.

[0114] The event classification module 406 is used to classify the target video clip using a pre-trained classification model to obtain a classification result; the classification result includes abnormal events and normal events.

[0115] The video storage module 408 is used to construct and store an association table according to the abnormal events and the target video segments corresponding to the abnormal events included in the classification results.

[0116] Furthermore, in one embodiment, the video acquisition module 402 is further configured to stitch the original video files using an image stitching algorithm to obtain an initial stitched video file; and to fuse the stitching seams of the initial stitched video file using a weighted average fusion algorithm to obtain a surround view video file.

[0117] Furthermore, in one embodiment, the event recognition module 406 is also used to split the surround video file according to preset time intervals to obtain multiple video segments when tracing the target event; for each video segment, the video segment is recognized in parallel according to each target event recognition condition to obtain recognition results of the video segment by each target event recognition condition; wherein, the recognition of each video segment is performed in parallel; and the recognition results of the video segment by each target event recognition condition are used as the recognition results corresponding to each target event recognition condition.

[0118] Furthermore, in one embodiment, the event recognition module 406 is also used to identify the surround video file according to the morphological structural elements corresponding to the target event to determine the moving target; use a tracking algorithm to track the behavior trajectory of the moving target, and determine the target video segment corresponding to the target event from the surround video file according to the behavior trajectory of the moving target.

[0119] Furthermore, in one embodiment, the surround view video processing device 400 also includes an abnormal alarm module, which is used to generate a safety report by analyzing the number of moving targets and behavior trajectories corresponding to each abnormal event when the classification result is that the target event is an abnormal event; determine the alarm method corresponding to the abnormal event, and alarm according to the alarm method.

[0120] Furthermore, in one embodiment, the video storage module 408 is also used to respond to a retrieval operation or a selection operation for key information of an abnormal event, determine a target video segment of the abnormal event corresponding to the key information according to the association table, and display the target video segment.

[0121] Each module in the above-mentioned surround video processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0122] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store original video files, surround video files, target video clips, target video clips and association table data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a surround video processing method is implemented.

[0123] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0125] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0126] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0127] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0128] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0129] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A surround video processing method, characterized in that: The method comprises: Acquire a surround view video file obtained by stitching together multiple original video files; the multiple original video files are acquired in real time by multiple cameras at preset angles and positions; According to at least one preset target event recognition condition, the surround view video file is recognized in parallel, and video extraction is performed on the surround view video file according to the recognition results corresponding to each of the target event recognition conditions to obtain a target video segment corresponding to each of the target event recognition conditions; Using a pre-trained classification model to classify the target video clip to obtain a classification result; the classification result includes abnormal events and normal events; According to the abnormal events included in the classification results and the target video segments corresponding to the abnormal events, a correlation table is constructed and stored.

2. The method according to claim 1, characterized in that The obtaining of a surround view video file obtained by splicing a plurality of original video files comprises: Using an image stitching algorithm to stitch the original video files together to obtain an initial stitched video file; A weighted average fusion algorithm is used to fuse the stitching seams of the initial stitching video file to obtain the surround view video file.

3. The method according to claim 1, characterized in that: The step of identifying the surround view video file in parallel according to at least one preset target event recognition condition, performing video extraction on the surround view video file according to the recognition results corresponding to each target event recognition condition, and obtaining a target video segment corresponding to each target event recognition condition includes: In the case of tracing the target event, the surround video file is split according to a preset time interval to obtain a plurality of video clips; For each video segment, the video segment is identified in parallel according to each of the target event identification conditions, and the identification results of the video segment by each of the target event identification conditions are obtained; wherein the identification of each of the video segments is performed in parallel; The recognition results of the video clips by the target event recognition conditions are used as the recognition results corresponding to the target event recognition conditions.

4. The method according to claim 1, characterized in that: The step of performing video extraction on the surround view video file according to the recognition results corresponding to each of the target event recognition conditions to obtain a target video segment corresponding to each of the target event recognition conditions includes: Identify the surround video file according to the morphological structure elements corresponding to the target event to determine the moving target; A tracking algorithm is used to track the behavior trajectory of the moving target, and a target video segment corresponding to the target event is determined from the surround video file according to the behavior trajectory of the moving target.

5. The method according to claim 4, characterized in that The method further comprises: classifying the target video clip using the pre-trained classification model to obtain the classification result; When the classification result is that the target event is an abnormal event, analyzing the number of moving targets and the behavior trajectories corresponding to each abnormal event to generate a safety report; Determine the alarm method corresponding to the abnormal event, and issue an alarm according to the alarm method.

6. The method according to claim 1, characterized in that After constructing a correlation table according to the abnormal events included in the classification results and the target video clips corresponding to the abnormal events and storing the correlation table, the method further includes: In response to a retrieval operation or a selection operation for key information of the abnormal event, a target video segment of the abnormal event corresponding to the key information is determined according to the association table, and the target video segment is displayed.

7. A surround video processing device, characterized in that: The device comprises: A video acquisition module is used to acquire a surround video file obtained by splicing multiple original video files; the multiple original video files are acquired in real time by multiple cameras at preset angles and positions; An event recognition module is used to recognize the surround view video file in parallel according to at least one preset target event recognition condition, and to extract the video from the surround view video file according to the recognition results corresponding to each target event recognition condition to obtain a target video segment corresponding to each target event recognition condition; An event classification module, used to classify the target video clip using a pre-trained classification model to obtain a classification result; the classification result includes abnormal events and normal events; The video storage module is used to construct an association table and store the association table according to the abnormal events included in the classification results and the target video clips corresponding to the abnormal events.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images