Video monitoring method, device, electronic equipment and storage medium

Abstract

The present invention provides a video monitoring method, device, electronic device and storage medium, including: based on the network graphics library drawing protocol, loading a three-dimensional virtual scene corresponding to a target monitoring area in a page; receiving a first input from a user, the first input being used to import multiple first monitoring videos in the target monitoring area; in response to the first input, displaying multiple first monitoring videos at multiple target positions in the three-dimensional virtual scene; multiple target positions correspond one to one with multiple first monitoring videos. The present invention can provide users with intuitive video images and simple view control, and at the same time, by adopting the network graphics library drawing technology, ensure that all monitoring video images integrated into the three-dimensional virtual scene can be played in parallel, with smooth images without lag, so that users can better perform global real-time video monitoring, greatly improving the efficiency of global real-time video monitoring.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video monitoring method, device, electronic equipment and storage medium. Background Art

[0002] In existing video surveillance systems, multiple split-lens surveillance images are usually displayed flatly on a monitor in the form of two-dimensional images, so that supervisors need to watch multiple split-lens images at the same time in the same interface. It is difficult to match the massive and scattered split-lens videos with their actual geographical locations. Especially for video surveillance of large spatial areas, it is impossible to intuitively identify the spatial area to which each surveillance video belongs, thereby failing to meet the monitoring needs of some users. The efficiency of global real-time video surveillance needs to be improved.

[0003] Therefore, how to better conduct video surveillance has become an urgent problem to be solved in the industry. Summary of the invention

[0004] The present invention provides a video monitoring method, device, electronic equipment and storage medium for better video monitoring.

[0005] The present invention provides a video monitoring method, comprising:

[0006] Based on the network graphics library drawing protocol, the three-dimensional virtual scene corresponding to the target monitoring area is loaded into the page;

[0007] Receive a first input from a user, where the first input is used to import multiple channels of first monitoring videos in the target monitoring area;

[0008] In response to the first input, multiple channels of the first monitoring videos are displayed at multiple target locations in the three-dimensional virtual scene; the multiple target locations correspond one-to-one to the multiple channels of the first monitoring videos.

[0009] According to a video monitoring method provided by the present invention, after displaying multiple channels of the first monitoring videos at multiple target positions in the three-dimensional virtual scene in response to the first input, the method further includes:

[0010] receiving a second input from a user, where the second input is used to perform image correction on each channel of the first surveillance video;

[0011] In response to the second input, generating a corrected multi-channel second surveillance video;

[0012] receiving a third input from a user, wherein the third input is used to splice each of the second surveillance videos with the three-dimensional virtual scene;

[0013] In response to the third input, a spliced ​​three-dimensional fusion scene is generated, and multiple channels of the second surveillance videos are displayed in the three-dimensional fusion scene.

[0014] According to a video monitoring method provided by the present invention, after displaying multiple channels of the first monitoring videos at multiple target positions in the three-dimensional virtual scene in response to the first input, the method further includes:

[0015] receiving a fourth input from the user;

[0016] In response to the fourth input, the first surveillance video at each of the target locations is traversed in sequence according to a preset video inspection path.

[0017] According to a video monitoring method provided by the present invention, the receiving of a second input from a user; and generating corrected multi-channel second monitoring videos in response to the second input, comprising:

[0018] Receive a first sub-input from a user, where the first sub-input is used to perform geometric distortion correction on all video frame images in each channel of the first surveillance video;

[0019] In response to the first sub-input, generating a corrected multi-channel third surveillance video;

[0020] receiving a second sub-input from a user, where the second sub-input is used to perform color correction on all video frame images of each channel of the third surveillance video;

[0021] In response to the second sub-input, generating a corrected multi-channel second surveillance video;

[0022] The second input includes the first sub-input and the second sub-input.

[0023] According to a video monitoring method provided by the present invention, receiving a third input from a user; generating a spliced ​​three-dimensional fusion scene in response to the third input, and displaying multiple channels of the second monitoring videos in the three-dimensional fusion scene, including:

[0024] receiving a third sub-input from a user, wherein the third sub-input is used to determine a target positioning viewing angle of the three-dimensional virtual scene;

[0025] In response to the third sub-input, generating a target three-dimensional virtual scene under the target positioning perspective;

[0026] Receive a fourth sub-input from a user, wherein the fourth sub-input is used to determine target parameters of each of the second surveillance videos in the target three-dimensional virtual scene; the target parameters at least include a display angle, a picture frame shape, and a picture size;

[0027] In response to the fourth sub-input, generating a spliced ​​three-dimensional fusion scene, and displaying multiple channels of the second monitoring videos in the three-dimensional fusion scene;

[0028] The third input includes the third sub-input and the fourth sub-input.

[0029] According to a video monitoring method provided by the present invention, after generating a spliced ​​3D fusion scene in response to the third input and displaying multiple channels of the second monitoring videos in the 3D fusion scene, the method further includes:

[0030] receiving a fifth input from the user;

[0031] In response to the fifth input, a target surveillance video is locked from the multiple second surveillance videos displayed in the three-dimensional fusion scene.

[0032] The present invention also provides a video monitoring device, comprising:

[0033] A loading module is used to load a three-dimensional virtual scene corresponding to the target monitoring area in the page based on a network graphics library drawing protocol;

[0034] An input module, used for receiving a first input from a user, wherein the first input is used for importing a plurality of first monitoring videos in the target monitoring area;

[0035] The display module is used to display multiple channels of the first monitoring videos at multiple target positions in the three-dimensional virtual scene in response to the first input; the multiple target positions correspond one-to-one to the multiple channels of the first monitoring videos.

[0036] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned video monitoring methods is implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the video monitoring method described in any one of the above is implemented.

[0038] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned video monitoring methods.

[0039] The video monitoring method, device, electronic device and storage medium provided by the present invention load a three-dimensional virtual scene corresponding to a target monitoring area in a page by adopting a network graphics library drawing protocol; and then by receiving a first input from a user, multiple first monitoring videos in the target monitoring area can be imported; in response to the first input, multiple first monitoring videos are displayed at multiple target positions in the loaded three-dimensional virtual scene; multiple target positions correspond to the multiple first monitoring videos one by one according to the monitoring areas to which the corresponding first monitoring videos belong, thereby merging the two-dimensional images of the monitoring videos of the target monitoring area into the corresponding three-dimensional virtual scene to achieve the effect of combining virtual and real display of the three-dimensional model of the target monitoring area with the real-time monitoring video, and can provide users with intuitive video images and simple view control. At the same time, by adopting the network graphics library drawing technology, the high-performance computing capability of a GPU can be used to complete real-time processing of video images on the front end of a web page without occupying CPU resources, ensuring that all monitoring video images fused into the three-dimensional virtual scene can be played in parallel, and the images are smooth and free of jamming, so that users can better perform global real-time video monitoring, and the efficiency of global real-time video monitoring is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 It is a flow chart of the video monitoring method provided by the present invention;

[0042] Figure 2 It is a structural schematic diagram of the video monitoring device provided by the present invention;

[0043] Figure 3 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Combine the following Figure 1-Figure 3 A video monitoring method, device, electronic device and storage medium of the present invention are described.

[0046] Figure 1 It is a flow chart of the video monitoring method provided by the present invention, such as Figure 1 As shown, it includes: step 110, step 120 and step 130.

[0047] Step 110, based on the network graphics library drawing protocol, loading the three-dimensional virtual scene corresponding to the target monitoring area in the page;

[0048] In this step, the network graphics library drawing protocol described in the embodiment of the present invention refers to the Web Graphics Library (WebGL) drawing protocol. Based on the technical advantage of WebGL being directly accessible on a browser and convenient and fast, in the embodiment of the present invention, according to the graphics library standard, the high-performance computing capability of the system graphics card GPU is used to process video images in real time, and image processing does not occupy CPU resources, and three-dimensional virtual scenes and models are efficiently rendered in the browser page.

[0049] The target monitoring area described in the embodiment of the present invention refers to a spatial area where a user needs to perform video monitoring, which can be an indoor office monitoring area or a monitoring area within an outdoor area, such as a building community monitoring area, a road traffic intersection monitoring area, etc.

[0050] In an embodiment of the present invention, by adopting the WebGL drawing protocol, the graphics can be rendered using the hardware acceleration of the underlying terminal device, so that the three-dimensional model can be displayed on the page without the support of the browser plug-in, with efficient graphics rendering speed. Therefore, according to the geographic information parameters of the target monitoring area, efficient graphics rendering can be performed according to the WebGL drawing protocol, and the three-dimensional virtual scene corresponding to the target monitoring area can be loaded in the network page.

[0051] Step 120, receiving a first input from a user, where the first input is used to import multiple channels of first surveillance videos in a target surveillance area;

[0052] In this step, the first monitoring video described in the embodiment of the present invention refers to the monitoring video collected in real time by each monitoring camera deployed in the target monitoring area.

[0053] The first input described in the embodiment of the present invention may be an operation of importing multiple channels of first surveillance videos into a target surveillance area.

[0054] In an embodiment of the present invention, in the interface for loading the three-dimensional virtual scene corresponding to the target monitoring area, a control for importing the first monitoring video can be pre-set, and multiple first monitoring videos in the target monitoring area can be imported by performing a first input operation on the control.

[0055] The first input may be specifically expressed in at least one of the following ways:

[0056] First, the second input may be a key input of an external input device (such as a mouse or a keyboard).

[0057] In this embodiment, multiple channels of first monitoring videos in the target monitoring area are imported by operating physical buttons of an external mouse and keyboard.

[0058] Secondly, the second input may be a touch input operation.

[0059] In this embodiment, receiving the first input from the user may be performed by receiving a touch input operation of the user on a related control in the three-dimensional virtual scene displayed on the terminal display screen, and importing multiple channels of first monitoring videos in the target monitoring area.

[0060] Of course, in other embodiments, the first input may also be in other forms, which may be determined according to actual needs, and the embodiments of the present invention are not limited to this.

[0061] Step 130 , in response to the first input, display multiple channels of first monitoring videos at multiple target locations in the three-dimensional virtual scene; the multiple target locations correspond one-to-one to the multiple channels of first monitoring videos.

[0062] Specifically, the target position in the embodiment of the present invention refers to a coordinate position determined in the three-dimensional virtual scene corresponding to the target monitoring area according to the geographical location information of the monitoring area to which each first monitoring video in the target monitoring area belongs.

[0063] In an embodiment of the present invention, multiple target positions correspond one-to-one to multiple first surveillance videos. That is to say, based on the geographical location of the surveillance area to which each first surveillance video belongs and the coordinate position of the geographical location in the three-dimensional virtual scene, the corresponding display position of each first surveillance video in the three-dimensional virtual scene of the target surveillance area can be determined.

[0064] For example, for monitoring road traffic intersections, according to the geographical location of each road intersection in the target monitoring area, the first monitoring video shot at each road intersection can be moved to the corresponding coordinate position in the three-dimensional virtual scene of the road monitoring area.

[0065] Furthermore, in response to the first input, corresponding multiple channels of first monitoring videos are displayed at multiple target positions in the three-dimensional virtual scene corresponding to the target monitoring area. Thus, by position matching and image fusion of one or more camera image sequence videos and the related three-dimensional virtual scenes, a new dynamic virtual scene or model of the target monitoring area scene is generated, thereby realizing the fusion of the virtual scene and the real-time monitoring video.

[0066] The video monitoring method of the embodiment of the present invention adopts the network graphics library drawing protocol to load the three-dimensional virtual scene corresponding to the target monitoring area in the page; then by receiving the first input of the user, multiple first monitoring videos in the target monitoring area can be imported; in response to the first input, multiple first monitoring videos are displayed at multiple target positions in the loaded three-dimensional virtual scene; multiple target positions correspond to the multiple first monitoring videos one by one according to the monitoring area to which the corresponding first monitoring videos belong, thereby merging the two-dimensional image of the monitoring video of the target monitoring area into the corresponding three-dimensional virtual scene to achieve the effect of combining the virtual and real display of the three-dimensional model of the target monitoring area and the real-time monitoring video, and can provide users with intuitive video images and simple view control. At the same time, by adopting the network graphics library drawing technology, the high-performance computing capability of the GPU can be used to complete the real-time processing of the video image on the front end of the web page without occupying CPU resources, ensuring that all monitoring video images integrated into the three-dimensional virtual scene can be played in parallel, and the images are smooth and free of jamming, which is convenient for users to better perform global real-time video monitoring, and greatly improves the efficiency of global real-time video monitoring.

[0067] Based on the content of the above embodiment, as an optional embodiment, in step 130, after displaying multiple channels of first monitoring videos at multiple target positions in the three-dimensional virtual scene in response to the first input, the method further includes:

[0068] Receive a second input from a user, where the second input is used to perform image correction on each channel of the first surveillance video;

[0069] In response to the second input, generating a corrected multi-channel second surveillance video;

[0070] receiving a third input from a user, where the third input is used to splice each second monitoring video with the three-dimensional virtual scene;

[0071] In response to the third input, a spliced ​​three-dimensional fusion scene is generated, and multiple channels of second surveillance videos are displayed in the three-dimensional fusion scene.

[0072] Specifically, the second input described in the embodiment of the present invention refers to an operation of performing image correction on each channel of first surveillance video, such as operations related to geometric distortion correction and image color correction.

[0073] The second surveillance video described in the embodiment of the present invention refers to a surveillance video obtained by performing image processing on all video frame images in the first surveillance video.

[0074] In the embodiment of the present invention, the second input is the same as the first input, and can also be expressed in at least one of the two modes described above for the first input, which will not be described in detail herein.

[0075] It should be noted that the process of camera imaging is actually the process of converting the points in the world coordinate system to the camera coordinate system, projecting them to obtain the image coordinate system, and then converting them to the pixel coordinate system. However, due to the distortion introduced by the lens precision and process, the straight line in the world coordinate system is no longer a straight line when converted to other coordinate systems, resulting in image distortion.

[0076] Furthermore, through the second input, image correction can be performed on each channel of the first surveillance video, and in response to the second input, a plurality of channels of corrected second surveillance videos are generated;

[0077] Based on the content of the above embodiment, as an optional embodiment, the above receiving the second input of the user; generating the corrected multi-channel second monitoring video in response to the second input includes:

[0078] Receive a first sub-input from a user, where the first sub-input is used to perform geometric distortion correction on all video frame images in each first monitoring video;

[0079] In response to the first sub-input, generating a corrected multi-channel third monitoring video;

[0080] Receive a second sub-input from a user, where the second sub-input is used to perform color correction on all video frame images in each channel of third surveillance video;

[0081] In response to the second sub-input, generating a corrected multi-channel second monitoring video;

[0082] The second input includes a first sub-input and a second sub-input.

[0083] Specifically, in an embodiment of the present invention, the second input may include a first sub-input and a second sub-input. The first sub-input and the second sub-input are the same as the first input, and may also be expressed in at least one of the two modes described for the first input, which will not be described in detail here.

[0084] The first sub-input described in the embodiment of the present invention refers to an operation of setting image geometric distortion correction parameters, which is used to perform geometric distortion correction on all video frame images in each channel of first surveillance video.

[0085] The second sub-input described in the embodiment of the present invention refers to the operation of setting image color correction parameters, which is used to perform color correction on all video frame images in each channel of the third surveillance video.

[0086] The third monitoring video described in the embodiment of the present invention refers to the monitoring video obtained after the first monitoring video is corrected for geometric distortion.

[0087] In an embodiment of the present invention, through the first sub-input of the user, the setting of geometric distortion correction parameters for all video frame images in each first surveillance video is completed, wherein the geometric distortion correction may mainly include X-axis direction correction, Y-axis direction correction, overall image scaling, and overall image offset along the X and Y axes;

[0088] Among them, the X-axis correction is used to straighten the straight lines that are bent in the X-axis direction in the image; the Y-axis correction is used to straighten the straight lines that are bent in the Y-axis direction in the image; the overall image scaling is used to correct the image display size; the overall image offset along the X and Y axes is used to correct the picture effect of the image display.

[0089] In an embodiment of the present invention, after receiving the first sub-input from the user, geometric distortion correction can be performed on all video frame images in each first surveillance video, and then in response to the first sub-input, multiple channels of third surveillance videos after correction can be generated in a three-dimensional virtual scene. More specifically, in an embodiment of the present invention, the setting of geometric distortion correction parameters for video images is completed through the first sub-input from the user, the CPU obtains the geometric distortion correction parameters set by the user, and transmits the geometric distortion correction parameters to the GPU. After the GPU obtains the geometric distortion correction parameters, it calls the shader GPUShader to perform geometric distortion correction on all video frame images in each first surveillance video, and renders and generates the corrected third surveillance video for each first surveillance video.

[0090] In an embodiment of the present invention, the setting of color correction parameters for all video frame images in each third surveillance video is completed through the user's second sub-input, wherein the color correction may include brightness adjustment, contrast adjustment, exposure adjustment, transparency adjustment, saturation adjustment, etc.

[0091] Furthermore, by receiving the second sub-input of the user, color correction can be performed on all video frame images in each channel of the third surveillance video, and then multiple channels of the second surveillance video after correction are generated in response to the second sub-input. More specifically, the color correction parameters of the video image are set through the second sub-input of the user, the CPU obtains the color correction parameters set by the user, and transmits the color correction parameters to the GPU. After the GPU obtains the color correction parameters, it calls the shader GPU Shader to perform color correction on all video frame images in each channel of the third surveillance video, and renders and generates multiple channels of the second surveillance video after correction.

[0092] It should be noted that in the embodiments of the present invention, all adjustment processes of the geometric distortion correction parameters and the color correction parameters are implemented manually in the preset editor in the three-dimensional virtual scene, thereby realizing the setting of the geometric distortion correction parameters and the color correction parameters of the video image according to the user's perception of the video image, which is conducive to the subsequent rapid realization of the stitching of multiple video sources and the full integration of the video color and the 3D model color.

[0093] In an embodiment of the present invention, by adopting WebGL technology and making full use of the GPU processing power, the correction parameters of each monitoring video can be flexibly adjusted to achieve geometric distortion correction and color correction of each first monitoring video in the three-dimensional virtual scene of the target monitoring area, which is conducive to the subsequent efficient splicing and fusion of multiple video sources and the three-dimensional virtual scene and color fusion.

[0094] The third input described in the embodiment of the present invention refers to the operation of splicing each second surveillance video with the three-dimensional virtual scene, wherein the third input may be the same as the first input, or may be expressed as at least one of the two methods described in the above first input, which will not be elaborated here.

[0095] The three-dimensional fusion scene described in the embodiment of the present invention refers to a scene picture in which the three-dimensional virtual scene of the target monitoring area and the pictures of each second monitoring video are spliced ​​together to integrate the three-dimensional virtual scene with the real-time dynamic video.

[0096] In an embodiment of the present invention, through the third input, the parameters of each second surveillance video at the target position in the three-dimensional virtual scene, such as video angle, video screen size, placement position, etc., are adjusted so that each second surveillance video is integrated into the surrounding three-dimensional model environment on a visual level, thereby realizing the splicing between each second surveillance video and the three-dimensional virtual scene.

[0097] Furthermore, in response to the third input, a spliced ​​three-dimensional fusion scene is generated, and multiple surveillance videos are displayed in the three-dimensional fusion scene, thereby further improving the effect of combining virtual and real display of the three-dimensional model of the target monitoring area and the real-time surveillance video, which is conducive to improving the user experience.

[0098] The method of the embodiment of the present invention performs image correction on each first monitoring video in the three-dimensional virtual scene of the target monitoring area, and splices the corrected monitoring video with the three-dimensional virtual scene, and displays multiple second monitoring videos in the spliced ​​three-dimensional fusion scene, thereby efficiently realizing the splicing, fusion and color fusion of multiple video sources in the target monitoring area and their corresponding three-dimensional virtual scenes, further improving the effect of the fusion display of the three-dimensional virtual scene and the real monitoring video, which is conducive to improving the efficiency of global real-time video monitoring.

[0099] Based on the content of the above embodiment, as an optional embodiment, in step 130, after displaying multiple channels of first monitoring videos at multiple target positions in the three-dimensional virtual scene in response to the first input, the method further includes:

[0100] receiving a fourth input from the user;

[0101] In response to the fourth input, the first surveillance video at each target location is traversed in sequence according to a preset video inspection path.

[0102] Specifically, the preset video inspection path described in the embodiment of the present invention refers to a video inspection path pre-planned by a user in a three-dimensional virtual scene of a target monitoring area.

[0103] The fourth input described in the embodiment of the present invention refers to an operation for performing video inspection path planning in a three-dimensional virtual scene. Its specific implementation method can be the same as the first input, and can also be expressed as at least one of the two methods described in the first input, which will not be described in detail here.

[0104] Furthermore, in this embodiment, the user can plan the video inspection path in the three-dimensional virtual scene in advance and complete the fourth input operation. By receiving the user's fourth input and responding to the fourth input, based on the built-in browser web 3D engine, it can control the browser perspective to automatically move along the planned path, and according to the preset video inspection path, traverse the first monitoring video at each target position in the three-dimensional virtual scene in turn, and see the content in the video at the same time, thereby achieving the purpose of inspecting the current three-dimensional virtual scene.

[0105] The method of the embodiment of the present invention can perform a global inspection of the video in the three-dimensional virtual scene of the target monitoring area by setting the video inspection path function, which is helpful to help users provide intuitive and accurate video data assistance for daily monitoring management and emergency handling, and improve the efficiency of global real-time video monitoring.

[0106] Based on the content of the above embodiment, as an optional embodiment, receiving a third input from a user; generating a spliced ​​3D fusion scene in response to the third input, and displaying multiple channels of second surveillance videos in the 3D fusion scene, including:

[0107] receiving a third sub-input from a user, where the third sub-input is used to determine a target positioning viewing angle of the three-dimensional virtual scene;

[0108] In response to the third sub-input, generating a target three-dimensional virtual scene under the target positioning perspective;

[0109] Receive a fourth sub-input from the user, where the fourth sub-input is used to determine target parameters of each second monitoring video in the target three-dimensional virtual scene; the target parameters include at least a display angle, a picture frame shape, and a picture size;

[0110] In response to the fourth sub-input, a spliced ​​three-dimensional fusion scene is generated, and multiple channels of second monitoring videos are displayed in the three-dimensional fusion scene;

[0111] The third input includes a third sub-input and a fourth sub-input.

[0112] Specifically, in an embodiment of the present invention, the third input may include a third sub-input and a fourth sub-input. The third sub-input and the fourth sub-input are the same as the first input, and may also be expressed in at least one of the two modes described for the first input, which will not be described in detail here.

[0113] The third sub-input described in the embodiment of the present invention refers to the operation of setting the target positioning perspective of the three-dimensional virtual scene, wherein the target positioning perspective can be a display perspective of the three-dimensional virtual scene customized by a user, or can be determined based on the monitoring perspective information of each second monitoring video, such as: the monitoring perspective of any monitoring video in each second monitoring video can be selected as the target positioning perspective.

[0114] The target three-dimensional virtual scene described in the embodiment of the present invention refers to a three-dimensional virtual scene of a target monitoring area under the target positioning perspective.

[0115] Furthermore, the setting of the target positioning perspective of the three-dimensional virtual scene is completed through the user's third sub-input, and then in response to the third sub-input, the target three-dimensional virtual scene under the target positioning perspective can be generated.

[0116] The fourth sub-input described in the embodiment of the present invention refers to the target parameters for determining each second monitoring video in the target three-dimensional virtual scene, wherein the target parameters at least include a display angle, a picture frame shape, and a picture size.

[0117] Among them, the display angle refers to the angle at which the two-dimensional picture of the second surveillance video is placed in the target three-dimensional virtual scene; the picture border shape refers to the picture border shape of the second surveillance video, which can be a square border, a circular border or an irregular border, and can be set according to user operation requirements; the picture size refers to the picture display size of the second surveillance video, which can be enlarged or scaled.

[0118] Further, receiving a fourth sub-input from the user, determining target parameters of each second monitoring video in the target three-dimensional virtual scene, including display angle, picture frame shape, and picture size, and then generating a spliced ​​three-dimensional fusion scene in response to the fourth sub-input, and displaying multiple second monitoring videos in the three-dimensional fusion scene;

[0119] More specifically, through the user's fourth sub-input, the setting of the target parameters of each second monitoring video in the target three-dimensional virtual scene is realized, the CPU obtains the target parameters of each second monitoring video set by the user, processes a frame of video frame images in the second monitoring video, thereby generating an image cropping template corresponding to each second monitoring video, and then transmits the image cropping template corresponding to each second monitoring video to the GPU. After the GPU obtains the image cropping template, it calls the shader GPU Shader to perform image cropping on all video frame images in the second monitoring video according to the image cropping template, so as to realize seamless splicing between the three-dimensional virtual scene and the monitoring video, and finally renders to generate the spliced ​​three-dimensional fusion scene, and displays multiple second monitoring videos in the three-dimensional fusion scene to perform video monitoring of the target monitoring area.

[0120] It should be noted that, in an embodiment of the present invention, all adjustment processes for adjusting the target parameters of each second surveillance video in the target three-dimensional virtual scene are implemented manually in a preset editor in the three-dimensional virtual scene, which can realize the setting of the target parameters of the surveillance video according to the user's perception of the video image, and can quickly realize the efficient splicing and fusion of multiple video sources and the three-dimensional virtual scene.

[0121] In one embodiment of the present invention, after generating a spliced ​​three-dimensional fusion scene and displaying multiple second surveillance videos in the three-dimensional fusion scene, if there is overlapping video screen content in the multiple second surveillance videos, a sixth input from the user for two second surveillance videos with overlapping video screen content is received, and the sixth input is used to crop the screen with overlapping content in any of the two second surveillance videos so that the seams of the two second surveillance videos overlap; in response to the sixth input, a cropped three-dimensional fusion scene is generated, thereby further improving the visual effect of the fusion display between the three-dimensional virtual scene and each surveillance video.

[0122] The method of the embodiment of the present invention adopts WebGL technology for processing and displaying surveillance videos, fully utilizes the GPU processing capability, and can quickly lock the three-dimensional virtual scene under the target positioning perspective through the user's third sub-input and fourth sub-input, and seamlessly splice each second surveillance video with the three-dimensional virtual scene based on the target parameters to generate a spliced ​​three-dimensional fusion scene, thereby quickly and efficiently completing the splicing and fusion between the three-dimensional virtual scene and each surveillance video.

[0123] Based on the content of the above embodiment, as an optional embodiment, after generating a spliced ​​3D fusion scene in response to a third input and displaying multiple channels of second surveillance videos in the 3D fusion scene, the method further includes:

[0124] receiving a fifth input from the user;

[0125] In response to the fifth input, a target surveillance video is locked from the multiple second surveillance videos displayed in the three-dimensional fusion scene.

[0126] Specifically, the target monitoring video described in the embodiment of the present invention refers to the monitoring video in the focus area of ​​the target monitoring area, which can be selected by the user to lock and enlarge one or more monitoring videos from multiple second monitoring videos.

[0127] The fifth input described in the embodiment of the present invention refers to the operation of the user selecting a target surveillance video from multiple second surveillance videos. Its specific implementation method can be the same as the first input, and can also be expressed as at least one of the two methods described in the first input, which will not be described in detail here.

[0128] Furthermore, in this embodiment, the user can set the target monitoring video in the three-dimensional virtual scene of the target monitoring area in advance and complete the fifth input operation. By receiving the user's fifth input and responding to the fifth input, based on the built-in browser web 3D engine, the 3D scene can be quickly switched to display, so as to lock the target monitoring video from multiple second monitoring videos and display the three-dimensional fusion scene under the corresponding perspective of the target monitoring video.

[0129] The method of the embodiment of the present invention can quickly display the three-dimensional virtual scene from the perspective of the target monitoring video with one click by setting a quick locking function for the monitoring of the key focus area, thereby realizing quick viewing of the monitoring video of the local key area in the three-dimensional virtual scene, which is beneficial for users to promptly discover emergencies in the key focus area and improve the efficiency of users in performing global real-time video monitoring.

[0130] The video monitoring device provided by the present invention is described below. The video monitoring device described below and the video monitoring method described above can be referred to each other.

[0131] Figure 2 Schematic diagram of the structure of the video monitoring device provided by the present invention. Figure 2 As shown, including:

[0132] The loading module 210 is used to load the three-dimensional virtual scene corresponding to the target monitoring area in the page based on the network graphics library drawing protocol;

[0133] An input module 220, configured to receive a first input from a user, wherein the first input is used to import multiple channels of first monitoring videos in the target monitoring area;

[0134] The display module 230 is used to display the multiple channels of the first monitoring videos at multiple target positions in the three-dimensional virtual scene in response to the first input; the multiple target positions correspond one-to-one to the multiple channels of the first monitoring videos.

[0135] The video surveillance device described in this embodiment can be used to execute the above-mentioned video surveillance method embodiment, and its principles and technical effects are similar, which will not be repeated here.

[0136] The video monitoring device of the embodiment of the present invention loads a three-dimensional virtual scene corresponding to the target monitoring area in a page by adopting a network graphics library drawing protocol; and then by receiving a first input from a user, multiple first monitoring videos in the target monitoring area can be imported; in response to the first input, multiple first monitoring videos are displayed at multiple target positions in the loaded three-dimensional virtual scene; multiple target positions correspond to the multiple first monitoring videos one by one according to the monitoring areas to which the corresponding first monitoring videos belong, thereby merging the two-dimensional images of the monitoring videos of the target monitoring area into the corresponding three-dimensional virtual scene to achieve an effect of combining virtual and real display of the three-dimensional model of the target monitoring area and the real-time monitoring video, and can provide users with intuitive video images and simple view control. At the same time, by adopting the network graphics library drawing technology, the high-performance computing capability of the GPU can be used to complete real-time processing of video images on the front end of the web page without occupying CPU resources, ensuring that all monitoring video images fused into the three-dimensional virtual scene can be played in parallel, and the images are smooth and free of jamming, which is convenient for users to better perform global real-time video monitoring, and greatly improves the efficiency of global real-time video monitoring.

[0137] Figure 3 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the video monitoring method provided by the above methods, the method comprising: loading a three-dimensional virtual scene corresponding to a target monitoring area in a page based on a network graphics library drawing protocol; receiving a first input from a user, the first input being used to import multiple first monitoring videos in the target monitoring area; in response to the first input, displaying multiple first monitoring videos at multiple target positions in the three-dimensional virtual scene; multiple target positions correspond one to one to multiple first monitoring videos.

[0138] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0139] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video surveillance method provided by the above-mentioned methods, which includes: based on a network graphics library drawing protocol, loading a three-dimensional virtual scene corresponding to a target monitoring area in a page; receiving a first input from a user, wherein the first input is used to import multiple first surveillance videos in the target monitoring area; in response to the first input, displaying multiple first surveillance videos at multiple target positions in the three-dimensional virtual scene; and multiple target positions correspond one-to-one to multiple first surveillance videos.

[0140] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the video surveillance method provided by the above-mentioned methods, the method comprising: loading a three-dimensional virtual scene corresponding to a target monitoring area in a page based on a network graphics library drawing protocol; receiving a first input from a user, the first input being used to import multiple first surveillance videos in the target monitoring area; in response to the first input, displaying multiple first surveillance videos at multiple target positions in the three-dimensional virtual scene; the multiple target positions corresponding one-to-one to the multiple first surveillance videos.

[0141] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video monitoring method, characterized in that: include: Based on the network graphics library drawing protocol, the three-dimensional virtual scene corresponding to the target monitoring area is loaded into the page; Receive a first input from a user, where the first input is used to import multiple channels of first monitoring videos in the target monitoring area; In response to the first input, displaying multiple channels of the first monitoring videos at multiple target locations in the three-dimensional virtual scene; The plurality of target locations correspond one to one with the plurality of the first monitoring videos; Wherein, after displaying the multiple channels of the first monitoring videos at the multiple target positions in the three-dimensional virtual scene in response to the first input, the method further includes: receiving a second input from a user, where the second input is used to perform image correction on each channel of the first surveillance video; In response to the second input, generating a corrected multi-channel second surveillance video; receiving a third input from a user, wherein the third input is used to splice each of the second surveillance videos with the three-dimensional virtual scene; In response to the third input, generate a spliced ​​three-dimensional fusion scene, and display multiple channels of the second surveillance videos in the three-dimensional fusion scene; The receiving of the third input from the user; generating a spliced ​​three-dimensional fusion scene in response to the third input, and displaying multiple channels of the second surveillance videos in the three-dimensional fusion scene, includes: receiving a third sub-input from a user, wherein the third sub-input is used to determine a target positioning viewing angle of the three-dimensional virtual scene; In response to the third sub-input, generating a target three-dimensional virtual scene under the target positioning perspective; Receive a fourth sub-input from a user, wherein the fourth sub-input is used to determine target parameters of each of the second surveillance videos in the target three-dimensional virtual scene; the target parameters at least include a display angle, a picture frame shape, and a picture size; In response to the fourth sub-input, generating a spliced ​​three-dimensional fusion scene, and displaying multiple channels of the second monitoring videos in the three-dimensional fusion scene; The third input includes the third sub-input and the fourth sub-input; After displaying the multiple channels of the first monitoring videos at multiple target positions in the three-dimensional virtual scene in response to the first input, the method further includes: receiving a fourth input from the user; In response to the fourth input, the first surveillance video at each of the target locations is traversed in sequence according to a preset video inspection path.

2. The video monitoring method according to claim 1, characterized in that: receiving a second input from a user; In response to the second input, generating a corrected multi-channel second surveillance video, including: Receive a first sub-input from a user, where the first sub-input is used to perform geometric distortion correction on all video frame images in each channel of the first surveillance video; In response to the first sub-input, generating a corrected multi-channel third surveillance video; receiving a second sub-input from a user, where the second sub-input is used to perform color correction on all video frame images of each channel of the third surveillance video; In response to the second sub-input, generating a corrected multi-channel second surveillance video; The second input includes the first sub-input and the second sub-input.

3. The video monitoring method according to claim 1, characterized in that: After receiving the third input from the user; generating a spliced ​​three-dimensional fusion scene in response to the third input, and displaying multiple channels of the second surveillance videos in the three-dimensional fusion scene, the method further includes: receiving a fifth input from the user; In response to the fifth input, a target surveillance video is locked from the multiple second surveillance videos displayed in the three-dimensional fusion scene.

4. A video monitoring device, characterized in that: include: A loading module is used to load a three-dimensional virtual scene corresponding to the target monitoring area in the page based on a network graphics library drawing protocol; An input module, used for receiving a first input from a user, wherein the first input is used for importing a plurality of first monitoring videos in the target monitoring area; A display module, configured to display the multiple channels of the first monitoring videos at multiple target locations in the three-dimensional virtual scene in response to the first input; the multiple target locations correspond one to one with the multiple channels of the first monitoring videos; The device is also specifically used for: receiving a second input from a user, where the second input is used to perform image correction on each channel of the first surveillance video; In response to the second input, generating a corrected multi-channel second surveillance video; receiving a third input from a user, wherein the third input is used to splice each of the second surveillance videos with the three-dimensional virtual scene; In response to the third input, generate a spliced ​​three-dimensional fusion scene, and display multiple channels of the second surveillance videos in the three-dimensional fusion scene; receiving a third sub-input from a user, wherein the third sub-input is used to determine a target positioning viewing angle of the three-dimensional virtual scene; In response to the third sub-input, generating a target three-dimensional virtual scene under the target positioning perspective; Receive a fourth sub-input from a user, wherein the fourth sub-input is used to determine target parameters of each of the second surveillance videos in the target three-dimensional virtual scene; the target parameters at least include a display angle, a picture frame shape, and a picture size; In response to the fourth sub-input, generating a spliced ​​three-dimensional fusion scene, and displaying multiple channels of the second monitoring videos in the three-dimensional fusion scene; The third input includes the third sub-input and the fourth sub-input; After displaying the multiple channels of the first monitoring videos at multiple target locations in the three-dimensional virtual scene in response to the first input, the method further includes: receiving a fourth input from the user; In response to the fourth input, the first surveillance video at each of the target locations is traversed in sequence according to a preset video inspection path.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the video monitoring method according to any one of claims 1 to 3 is implemented.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video monitoring method according to any one of claims 1 to 3 is implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the video monitoring method according to any one of claims 1 to 3 is implemented.

Images