Intelligent analysis web page real-time visualization method, device, equipment and storage medium

Abstract

The present invention relates to the field of data processing technology, and discloses a method, device, equipment and storage medium for real-time visualization of intelligent analysis web pages. The method obtains a video stream collected by a target camera and generates a real-time analysis result. According to the real-time analysis result and the hardware configuration status information of a visualization monitoring terminal, the reasonable configuration and maximum utilization of the available encoding resources of the encoding server and the decoding capability of the visualization monitoring terminal are considered. The real-time analysis content and the video stream of the target high-synchronization visualization monitoring terminal and the low-synchronization visualization monitoring terminal are video encoded, and then encapsulated and transmitted with the real-time analysis content of the non-target high-synchronization visualization monitoring terminal, so that the decoding of a part of the real-time analysis content is executed on the visualization monitoring terminal instead of the decoding of all the real-time analysis content, which can balance the processing load of the server and the web page terminal, improve the processing efficiency of the server and the visualization effect of playing the monitoring video and the analysis content on the web page.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for intelligent analysis of real-time visualization of web pages. Background Art

[0002] Surveillance video analysis visualization is a technology that analyzes the video information collected by the camera and presents the analysis results in real time on the video. By displaying the analysis results in real time, it can improve the efficiency of video surveillance and management in the fields of security monitoring, traffic management and industrial production, enhance the safety of production and life, and help managers optimize decision-making basis.

[0003] At present, there are usually two ways to present the analysis results on the video: first, the collected surveillance video and the video analysis results (usually analysis text) are encoded on the server side, and the encoded video stream data is stored. The web page generates the video by calling the encoded video stream data. The synchronization of the surveillance video and the video analysis results played on the web page can be improved by directly embedding the video analysis results into the video frame. However, due to the expensive and limited encoding resources, this implementation method has the problem of tight server encoding resources when facing the parallel processing scenario of visually presenting multiple surveillance videos or multiple analysis contents of a single surveillance video, which limits the parallel processing of the monitoring system. The first is to encode the surveillance video separately on the server side, and store the encoded video together with the video analysis results. The web page can decode the video and visualize the video analysis results on the terminal by calling the encoded video and video analysis results, which can reduce the server encoding resources consumed by the server side to encode the video analysis results and the surveillance video together. However, due to the differences in the hardware configuration levels of the terminals, distributing the decoding load to the terminals will cause some terminals with lower hardware configuration levels to have abnormalities during decoding, especially in the parallel processing scenario of visualizing multiple analysis contents of a single surveillance video, which can easily lead to the surveillance video and the video analysis results being out of sync. In summary, the existing surveillance video analysis visualization technology has a contradiction in the hardware resource configuration between the server side and the web terminal, which makes the server processing efficiency low and the web side playing surveillance video and analysis content poor visualization effect when presenting the analysis results on the video.

[0004] Therefore, how to balance the processing efficiency and presentation effect of surveillance video analysis visualization and provide a better video analysis visualization experience is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a method, device, equipment and storage medium for intelligent analysis of real-time visualization of web pages, aiming to solve at least one of the above-mentioned technical problems.

[0006] To achieve the above object, the present invention provides a method for intelligently analyzing web pages in real time, comprising the following steps:

[0007] Acquire a video stream captured by a target camera, perform intelligent analysis on each video frame of the video stream, and obtain a real-time analysis result of each video frame in the video stream; wherein the real-time analysis result includes real-time analysis contents corresponding to a plurality of visual monitoring terminals respectively;

[0008] According to the real-time analysis result, the video stream and a plurality of real-time analysis contents of each video frame are encapsulated, and the encapsulated video analysis stream is sent to a streaming media server, so that the streaming media server transcodes the video analysis stream into video stream data of a target type;

[0009] When a video stream pulling instruction is received, the video stream data obtained by transcoding is pulled at the visual monitoring terminal, and the video stream and real-time analysis content in the video stream data are extracted;

[0010] A playback video is generated based on the video stream, and the analysis result is drawn on a video frame corresponding to a timestamp in the playback video to obtain a video analysis visualization result.

[0011] Optionally, the steps for obtaining the video stream captured by the target camera include:

[0012] Obtain identification information of the target camera, and match the camera RTSP address of the target camera in the camera address list based on the identification information;

[0013] The camera RTSP address is subjected to an availability test, and after the availability test is passed, the camera RTSP address is used to pull the video stream of the target monitoring area collected by the target camera.

[0014] Optionally, performing intelligent analysis on each video frame of the video stream to obtain a real-time analysis result of each video frame in the video stream specifically includes:

[0015] The video streams are respectively input into a plurality of intelligent analysis models corresponding to different analysis contents for the target monitoring area, and real-time analysis contents of each video frame output by the plurality of intelligent analysis models are obtained;

[0016] Based on the similarity matching results of the analysis content of each intelligent analysis model and the monitoring content of each visual monitoring terminal, the association relationship between the real-time analysis content output by the intelligent analysis model and the visual monitoring terminal is determined, and real-time analysis content mapping information consisting of a plurality of real-time analysis contents with association relationship and visual monitoring terminals is established;

[0017] According to the real-time analysis content mapping information corresponding to each intelligent analysis model, a real-time analysis result of each video frame in the video stream is generated.

[0018] Optionally, according to the real-time analysis result, encapsulating the video stream and a plurality of real-time analysis contents of each video frame, and sending the encapsulated video analysis stream to a streaming media server so that the streaming media server transcodes the video analysis stream into a target type of video stream data step, specifically comprising:

[0019] According to the hardware configuration status information and application analysis content of several visual monitoring terminals, the visual monitoring terminals are divided into high-synchronization visual monitoring terminals and low-synchronization visual monitoring terminals; wherein the high-synchronization visual monitoring terminals include target high-synchronization visual monitoring terminals and non-target high-synchronization visual monitoring terminals;

[0020] Encode the real-time analysis content corresponding to the target high-synchronization visual monitoring terminal and the low-synchronization visual monitoring terminal and each video frame of the video stream, and encapsulate the encoded video stream and several real-time analysis contents of the non-target high-synchronization visual monitoring terminals into NAL frames using JSON, merge the NAL frames into ES code streams for packaging and MPEGES packaging;

[0021] The video analysis stream obtained by encapsulating MPEGES is sent to a streaming media server using RTP, so that the streaming media server transcodes the video analysis stream into FLV video stream data.

[0022] Optionally, the step of dividing the visual monitoring terminals into high-synchronization visual monitoring terminals and low-synchronization visual monitoring terminals according to the hardware configuration status information and application analysis content of the plurality of visual monitoring terminals specifically includes:

[0023] Acquire hardware configuration status information of a plurality of visual monitoring terminals, and divide the visual monitoring terminals into high-synchronization visual monitoring terminals and low-synchronization visual monitoring terminals according to the hardware configuration status information and preset hardware configuration status requirements required for performing text and video synchronization;

[0024] Extracting text information of real-time analysis content corresponding to each high-synchronous visual monitoring terminal, and calculating, based on the number of text display words and the text display format in the text information, a quantized value of encoding performance consumption when encoding and a quantized value of decoding performance consumption when decoding the real-time analysis content corresponding to each high-synchronous visual monitoring terminal according to a preset conversion relationship;

[0025] The first constraint condition is that the sum of the quantized values ​​of the encoding performance consumption of the real-time analysis content corresponding to the selected target high-synchronous visual monitoring terminals in the previous monitoring cycle is less than the quantized value of the encoding standard performance consumption of the encoding server, and the second constraint condition is that the sum of the quantized values ​​of the decoding performance consumption of the real-time analysis content corresponding to the selected target high-synchronous visual monitoring terminals in the previous monitoring cycle is less than the quantized value of the decoding standard performance consumption of each visual monitoring terminal; the optimization goal is to maximize the number of selected target high-synchronous visual monitoring terminals, optimize the selection strategy of the target high-synchronous visual monitoring terminals in the current monitoring cycle, and determine the target high-synchronous visual monitoring terminals.

[0026] Optionally, when a video stream pulling instruction is received, the video stream data obtained by transcoding is pulled at the visual monitoring terminal, and the video stream and real-time analysis content steps in the video stream data are extracted, specifically including:

[0027] When a video stream pull instruction is received, the FLV video stream data obtained by transcoding is pulled at the visual monitoring terminal; wherein the video stream data includes real-time analysis content of a non-target high-synchronous visual monitoring terminal and video stream encoding data with real-time analysis content corresponding to a target high-synchronous visual monitoring terminal and a low-synchronous visual monitoring terminal;

[0028] The real-time analysis content of the non-target high-synchronous visual monitoring terminal and the video stream encoding data are extracted from the video stream data, the video stream encoding data is used as the video generation data, and the real-time analysis content of the non-target high-synchronous visual monitoring terminal is used as the video rendering data.

[0029] Optionally, generating a playback video based on the video stream, drawing the analysis result on a video frame corresponding to a timestamp in the playback video, and obtaining a step of visualizing the video analysis result specifically includes:

[0030] The video generation data is sent to a decoder of a visualization monitoring terminal, and the decoder is used to generate a decoded video containing real-time analysis content corresponding to the original video and the target high-synchronization visualization monitoring terminal and the low-synchronization visualization monitoring terminal;

[0031] Based on the video rendering data, the real-time analysis content of the non-target high-synchronization visualization monitoring terminal is drawn on the decoded video to obtain a video analysis visualization result.

[0032] In addition, in order to achieve the above-mentioned purpose, the present invention also provides a device for intelligently analyzing web pages in real time and visualizing the web pages, comprising:

[0033] An acquisition module is used to acquire a video stream collected by a target camera, perform intelligent analysis on each video frame of the video stream, and obtain a real-time analysis result of each video frame in the video stream; wherein the real-time analysis result includes real-time analysis contents corresponding to a plurality of visual monitoring terminals respectively;

[0034] An encapsulation module is used to encapsulate the video stream and a plurality of real-time analysis contents of each video frame according to the real-time analysis result, and send the encapsulated video analysis stream to a streaming media server so that the streaming media server transcodes the video analysis stream into video stream data of a target type;

[0035] An extraction module, for pulling the video stream data obtained by transcoding at a visual monitoring terminal when receiving a video stream pulling instruction, and extracting the video stream and real-time analysis content from the video stream data;

[0036] A generation module is used to generate a playback video based on the video stream, draw the analysis result on a video frame corresponding to a timestamp in the playback video, and obtain a video analysis visualization result.

[0037] In addition, in order to achieve the above-mentioned purpose, the present invention also provides an intelligent analysis web page real-time visualization device, which includes: a memory, a processor, and an intelligent analysis web page real-time visualization program stored in the memory and executable on the processor, and when the intelligent analysis web page real-time visualization program is executed by the processor, the steps of the intelligent analysis web page real-time visualization method as described above are implemented.

[0038] In addition, in order to achieve the above-mentioned purpose, the present invention also provides a storage medium, on which is stored a smart analysis web page real-time visualization program, and when the smart analysis web page real-time visualization program is executed by a processor, the steps of the above-mentioned smart analysis web page real-time visualization method are implemented.

[0039] The beneficial effects of the present invention are as follows: a method, device, equipment and storage medium for real-time visualization of intelligent analysis web pages are proposed, by acquiring the video stream collected by the target camera and generating real-time analysis results, the visualization monitoring terminal is divided into high-synchronization visualization monitoring terminals and low-synchronization visualization monitoring terminals with different hardware configurations according to the real-time analysis results and the hardware configuration status information of the visualization monitoring terminal, and by considering the reasonable configuration and maximum utilization of the available encoding resources of the encoding server and the decoding capabilities of the visualization monitoring terminal, the target high-synchronization visualization monitoring terminal and the non-target high-synchronization visualization monitoring terminal are extracted from the high-synchronization visualization monitoring terminal, and then the real-time analysis content and video stream of the target high-synchronization visualization monitoring terminal and the low-synchronization visualization monitoring terminal are video encoded, and then the video encoding and the real-time analysis content of the non-target high-synchronization visualization monitoring terminal are encapsulated and transmitted, so that the decoding of a part of the real-time analysis content is performed on the visualization monitoring terminal instead of the decoding of all the real-time analysis content, the processing load of the server and the web terminal is balanced, and the server processing efficiency and the visualization effect of playing monitoring video and analysis content on the web side are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a flow chart of an embodiment of a method for intelligently analyzing web pages and visualizing them in real time according to the present invention;

[0042] Figure 3 The present invention is a block diagram of a device for intelligently analyzing and visualizing web pages in real time according to an embodiment of the present invention.

[0043] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0046] like Figure 1As shown, the device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0047] Those skilled in the art will understand that Figure 1 The structure of the device shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0048] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an intelligent analysis web page real-time visualization program.

[0049] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the intelligent analysis web page real-time visualization program stored in the memory 1005, and perform the following operations:

[0050] Acquire a video stream captured by a target camera, perform intelligent analysis on each video frame of the video stream, and obtain a real-time analysis result of each video frame in the video stream; wherein the real-time analysis result includes real-time analysis contents corresponding to a plurality of visual monitoring terminals respectively;

[0051] According to the real-time analysis result, the video stream and a plurality of real-time analysis contents of each video frame are encapsulated, and the encapsulated video analysis stream is sent to a streaming media server, so that the streaming media server transcodes the video analysis stream into video stream data of a target type;

[0052] When a video stream pulling instruction is received, the video stream data obtained by transcoding is pulled at the visual monitoring terminal, and the video stream and real-time analysis content in the video stream data are extracted;

[0053] A playback video is generated based on the video stream, and the analysis result is drawn on a video frame corresponding to a timestamp in the playback video to obtain a video analysis visualization result.

[0054] The specific embodiments of the present invention applied to the device are basically the same as the embodiments of the following application intelligent analysis method for real-time visualization of web pages, and will not be described in detail here.

[0055] The embodiment of the present invention provides a method for real-time visualization of intelligent analysis web pages, referring to Figure 2 , Figure 2 It is a flow chart of an embodiment of the method for intelligently analyzing web pages and visualizing them in real time according to the present invention.

[0056] In this embodiment, the method for real-time visualization of intelligent analysis web pages includes the following steps:

[0057] S100: Obtain a video stream captured by a target camera, perform intelligent analysis on each video frame of the video stream, and obtain a real-time analysis result of each video frame in the video stream; wherein the real-time analysis result includes real-time analysis contents corresponding to a plurality of visual monitoring terminals respectively;

[0058] S200: encapsulating the video stream and a plurality of real-time analysis contents of each video frame according to the real-time analysis result, and sending the encapsulated video analysis stream to a streaming media server, so that the streaming media server transcodes the video analysis stream into video stream data of a target type;

[0059] S300: when a video stream pulling instruction is received, the video stream data obtained by transcoding is pulled at the visual monitoring terminal, and the video stream and real-time analysis content in the video stream data are extracted;

[0060] S400: Generate a playback video based on the video stream, draw the analysis result on a video frame corresponding to a timestamp in the playback video, and obtain a video analysis visualization result.

[0061] It should be noted that, at present, there are usually two ways to present the analysis results on the video: first, the collected surveillance video and the video analysis results (usually analysis text) are encoded on the server side, and the encoded video stream data is stored. The web page generates the video by calling the encoded video stream data. The synchronization of the surveillance video and the video analysis results played on the web page can be improved by directly embedding the video analysis results into the video frame. However, due to the expensive and limited encoding resources, this implementation method has the problem of tight server encoding resources when facing the parallel processing scenario of visual presentation of multiple surveillance videos or multiple analysis contents of a single surveillance video, which limits the parallelization of the monitoring system. The processing capacity of the monitoring video is reduced, which affects the visual presentation effect; secondly, the monitoring video is encoded separately on the server side, and the encoded video is stored together with the video analysis results. The web page calls the encoded video and the video analysis results to perform video decoding and visual presentation of the video analysis results on the terminal, which can reduce the server encoding resources consumed by the server side to encode the video analysis results and the monitoring video together. However, due to the differences in the hardware configuration levels of the terminals, distributing the decoding load to the terminals will cause some terminals with lower hardware configuration levels to have abnormalities during decoding, especially in the parallel processing scenario of visual presentation of multiple analysis contents of a single monitoring video, which is easy to cause the monitoring video and the video analysis results to be out of sync. In summary, the existing monitoring video analysis visualization technology has a contradiction in the hardware resource configuration between the server and the web terminal, which makes the server processing efficiency low and the web page playing the monitoring video and the analysis content The visual effect of the content is poor.

[0062] In order to solve the above problems, this embodiment obtains the video stream collected by the target camera and generates a real-time analysis result. According to the real-time analysis result and the hardware configuration status information of the visual monitoring terminal, the visual monitoring terminal is divided into a high-synchronization visual monitoring terminal and a low-synchronization visual monitoring terminal with different hardware configurations. By considering the reasonable configuration and maximum utilization of the available encoding resources of the encoding server and the decoding capability of the visual monitoring terminal, the target high-synchronization visual monitoring terminal and the non-target high-synchronization visual monitoring terminal are extracted from the high-synchronization visual monitoring terminal, and then the real-time analysis content and video stream of the target high-synchronization visual monitoring terminal and the low-synchronization visual monitoring terminal are video encoded, and then the video encoding and the real-time analysis content of the non-target high-synchronization visual monitoring terminal are packaged and transmitted, so that the decoding of a part of the real-time analysis content instead of the decoding of all the real-time analysis content is performed on the visual monitoring terminal, so as to balance the processing load of the server and the web terminal, and improve the server processing efficiency and the visualization effect of playing monitoring video and analysis content on the web side.

[0063] In a preferred embodiment, the step of obtaining the video stream captured by the target camera specifically includes:

[0064] S110: Acquire identification information of a target camera, and match a camera RTSP address of the target camera in a camera address list based on the identification information;

[0065] S120: Perform an availability test on the camera RTSP address. After the availability test passes, use the camera RTSP address to pull the video stream of the target monitoring area collected by the target camera.

[0066] On this basis, the video stream is intelligently analyzed for each video frame to obtain a real-time analysis result of each video frame in the video stream, specifically comprising:

[0067] S130: inputting the video stream into a plurality of intelligent analysis models corresponding to different analysis contents for the target monitoring area, and obtaining real-time analysis contents of each video frame output by the plurality of intelligent analysis models;

[0068] S140: Based on the similarity matching results of the analysis content of each intelligent analysis model and the monitoring content of each visual monitoring terminal, determine the association relationship between the real-time analysis content output by the intelligent analysis model and the visual monitoring terminal, and establish real-time analysis content mapping information consisting of a plurality of real-time analysis contents with association relationship and visual monitoring terminals;

[0069] S150: Generate a real-time analysis result for each video frame in the video stream according to the real-time analysis content mapping information corresponding to each intelligent analysis model.

[0070] In this embodiment, after obtaining the video stream of the target monitoring area captured by the target camera, the video stream is respectively input into several intelligent analysis models corresponding to different analysis contents for the target monitoring area, and real-time analysis results including real-time analysis contents are generated, and a mapping relationship with the visual monitoring terminal is established for subsequent calls. It should be noted that the intelligent analysis module is configured as an artificial intelligence model for identifying different analysis contents in the video frame (such as production violation identification, entry and exit face recognition, etc.), and provides the monitoring video with analysis content requirements for viewing by different visual monitoring terminals.

[0071] In a preferred embodiment, according to the real-time analysis result, the video stream and several real-time analysis contents of each video frame are encapsulated, and the encapsulated video analysis stream is sent to a streaming media server so that the streaming media server transcodes the video analysis stream into a target type of video stream data step, specifically including:

[0072] S210: According to the hardware configuration status information and application analysis content of the plurality of visual monitoring terminals, the visual monitoring terminals are divided into high-synchronization visual monitoring terminals and low-synchronization visual monitoring terminals; wherein the high-synchronization visual monitoring terminals include target high-synchronization visual monitoring terminals and non-target high-synchronization visual monitoring terminals;

[0073] S220: Encode the real-time analysis content corresponding to the target high-synchronization visual monitoring terminal and the low-synchronization visual monitoring terminal and each video frame of the video stream, and encapsulate the encoded video stream and several real-time analysis contents of the non-target high-synchronization visual monitoring terminals into NAL frames using JSON, merge the NAL frames into ES code streams for packaging and MPEGES packaging;

[0074] S230: Using RTP to send the video analysis stream obtained by encapsulating MPEGES to a streaming server, so that the streaming server transcodes the video analysis stream into FLV video stream data.

[0075] Furthermore, according to the hardware configuration status information and application analysis content of the plurality of visual monitoring terminals, the visual monitoring terminals are divided into high-synchronization visual monitoring terminals and low-synchronization visual monitoring terminals, specifically including:

[0076] S211: Acquire hardware configuration status information of a plurality of visual monitoring terminals, and divide the visual monitoring terminals into high-synchronization visual monitoring terminals and low-synchronization visual monitoring terminals according to the hardware configuration status information and preset hardware configuration status requirements required for performing text and video synchronization;

[0077] S212: extracting text information of the real-time analysis content corresponding to each high-synchronous visual monitoring terminal, and calculating the encoding performance cost quantization value of the real-time analysis content corresponding to each high-synchronous visual monitoring terminal when encoding and the decoding performance cost quantization value when decoding according to a preset conversion relationship based on the number of text display words and the text display format in the text information;

[0078] S213: The first constraint condition is that the sum of the quantized values ​​of the encoding performance consumption of the real-time analysis content of the selected target high-synchronous visual monitoring terminals in the previous monitoring cycle is less than the quantized value of the encoding standard performance consumption of the encoding server; the second constraint condition is that the sum of the quantized values ​​of the decoding performance consumption of the real-time analysis content of the selected target high-synchronous visual monitoring terminals in the previous monitoring cycle is less than the quantized value of the decoding standard performance consumption of each visual monitoring terminal; the optimization goal is to maximize the number of selected target high-synchronous visual monitoring terminals, optimize the selection strategy of the target high-synchronous visual monitoring terminals in the current monitoring cycle, and determine the target high-synchronous visual monitoring terminals.

[0079] In this embodiment, according to the real-time analysis results and the hardware configuration status information of the visualization monitoring terminal, the visualization monitoring terminal is divided into high-synchronization visualization monitoring terminals and low-synchronization visualization monitoring terminals with different hardware configurations, and by considering the reasonable configuration and maximum utilization of the available encoding resources of the encoding server and the decoding capabilities of the visualization monitoring terminal, the target high-synchronization visualization monitoring terminal and the non-target high-synchronization visualization monitoring terminal are extracted from the high-synchronization visualization monitoring terminal. Thus, through the optimal solution method, the visualization monitoring terminal is divided into low-synchronization visualization monitoring terminals, target high-synchronization visualization monitoring terminals and non-target high-synchronization visualization monitoring terminals, so that when the analysis content of the monitoring video is visualized in the current monitoring cycle, the corresponding encoding, decoding and packaging actions are performed according to the division results of the visualization monitoring terminal, so as to improve the server processing efficiency and the visualization effect of the monitoring video and analysis content played on the web page.

[0080] In a preferred embodiment, when a video stream pulling instruction is received, the video stream data obtained by transcoding is pulled at the visual monitoring terminal, and the video stream and real-time analysis content steps in the video stream data are extracted, specifically including:

[0081] S310: when a video stream pull instruction is received, the FLV video stream data obtained by transcoding is pulled at the visual monitoring terminal; wherein the video stream data includes real-time analysis content of a non-target high-synchronous visual monitoring terminal and video stream encoding data having real-time analysis content corresponding to a target high-synchronous visual monitoring terminal and a low-synchronous visual monitoring terminal;

[0082] S320: extracting the real-time analysis content of the non-target high-synchronous visual monitoring terminal and the video stream encoding data in the video stream data, using the video stream encoding data as the video generation data, and using the real-time analysis content of the non-target high-synchronous visual monitoring terminal as the video rendering data.

[0083] On this basis, a playback video is generated based on the video stream, and the analysis result is drawn on a video frame corresponding to a timestamp in the playback video to obtain a step of visualizing the video analysis result, specifically including:

[0084] S410: sending the video generation data to a decoder of a visualization monitoring terminal, and using the decoder to generate a decoded video including the original video and real-time analysis content corresponding to the target high-synchronization visualization monitoring terminal and the low-synchronization visualization monitoring terminal;

[0085] S420: Based on the video drawing data, draw the real-time analysis content of the non-target high-synchronization visualization monitoring terminal on the decoded video to obtain a video analysis visualization result.

[0086] In this embodiment, the real-time analysis content and video stream of the target high-synchronization visualization monitoring terminal and the low-synchronization visualization monitoring terminal are encoded, and then the video encoding and the real-time analysis content of the non-target high-synchronization visualization monitoring terminal are packaged and transmitted, so that the decoding of a part of the real-time analysis content instead of the decoding of all the real-time analysis content is performed on the visualization monitoring terminal, so as to balance the processing load of the server and the web terminal, and improve the server processing efficiency and the visualization effect of playing the monitoring video and the analysis content on the web side.

[0087] Reference Figure 3 , Figure 3 It is a structural block diagram of an embodiment of the device for intelligently analyzing web pages and visualizing them in real time according to the present invention.

[0088] like Figure 3 As shown, the intelligent analysis webpage real-time visualization device proposed in the embodiment of the present invention includes:

[0089] The acquisition module 10 is used to acquire the video stream collected by the target camera, perform intelligent analysis on each video frame of the video stream, and obtain the real-time analysis result of each video frame in the video stream; wherein the real-time analysis result includes the real-time analysis content corresponding to a plurality of visual monitoring terminals respectively;

[0090] The encapsulation module 20 is used to encapsulate the video stream and a plurality of real-time analysis contents of each video frame according to the real-time analysis result, and send the encapsulated video analysis stream to the streaming media server so that the streaming media server transcodes the video analysis stream into the target type of video stream data;

[0091] The extraction module 30 is used to pull the video stream data obtained by transcoding at the visual monitoring terminal when receiving a video stream pulling instruction, and extract the video stream and real-time analysis content from the video stream data;

[0092] The generating module 40 is used to generate a playback video based on the video stream, draw the analysis result on a video frame corresponding to a timestamp in the playback video, and obtain a video analysis visualization result.

[0093] Other embodiments or specific implementations of the device for intelligently analyzing web pages and visualizing them in real time according to the present invention may refer to the above-mentioned method embodiments and will not be described in detail here.

[0094] In addition, the present invention also proposes a real-time visualization device for intelligent analysis web pages, which includes: a memory, a processor, and a real-time visualization program for intelligent analysis web pages stored in the memory and executable on the processor, and when the real-time visualization program for intelligent analysis web pages is executed by the processor, the steps of the real-time visualization method for intelligent analysis web pages as described above are implemented.

[0095] The specific implementation of the intelligent analysis web page real-time visualization device of the present application is basically the same as the embodiments of the above-mentioned intelligent analysis web page real-time visualization method, and will not be repeated here.

[0096] In addition, the present invention also proposes a readable storage medium, the readable storage medium includes a computer readable storage medium, on which is stored an intelligent analysis web page real-time visualization program. The readable storage medium may be Figure 1 The memory 1005 in the terminal may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The readable storage medium includes a number of instructions for enabling an intelligent analysis web page real-time visualization device with a processor to execute the intelligent analysis web page real-time visualization method described in each embodiment of the present invention.

[0097] The specific implementation methods in the readable storage medium of the present application are basically the same as the embodiments of the above-mentioned intelligent analysis web page real-time visualization method, and will not be repeated here.

[0098] It is understood that, in the description of this specification, the description with reference to the terms "one embodiment", "another embodiment", "other embodiments", or "first to Nth embodiments" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0099] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0100] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0102] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for real-time visualization of intelligent analysis web pages, characterized in that: The following steps are involved: Acquire a video stream captured by a target camera, perform intelligent analysis on each video frame of the video stream, and obtain a real-time analysis result of each video frame in the video stream; wherein the real-time analysis result includes real-time analysis contents corresponding to a plurality of visual monitoring terminals respectively; According to the real-time analysis result, the video stream and several real-time analysis contents of each video frame are encapsulated, and the encapsulated video analysis stream is sent to the streaming media server, so that the streaming media server transcodes the video analysis stream into the target type of video stream data; specifically including: according to the hardware configuration status information and application analysis content of several visual monitoring terminals, the visual monitoring terminals are divided into high-synchronicity visual monitoring terminals and low-synchronicity visual monitoring terminals; wherein the high-synchronicity visual monitoring terminals include target high-synchronicity visual monitoring terminals and non-target high-synchronicity visual monitoring terminals; the real-time analysis contents corresponding to the target high-synchronicity visual monitoring terminals and the low-synchronicity visual monitoring terminals are encoded with each video frame of the video stream, and the encoded video stream and several real-time analysis contents of the non-target high-synchronicity visual monitoring terminals are encapsulated into NAL frames using JSON, and the NAL frames are merged into ES code streams for packaging and MPEGES packaging; the video analysis stream obtained by MPEGES packaging is sent to the streaming media server using RTP, so that the streaming media server transcodes the video analysis stream into FLV video stream data; When a video stream pulling instruction is received, the video stream data obtained by transcoding is pulled at the visual monitoring terminal, and the video stream and real-time analysis content in the video stream data are extracted; A playback video is generated based on the video stream, and the analysis result is drawn on a video frame corresponding to a timestamp in the playback video to obtain a video analysis visualization result.

2. The method for real-time visualization of intelligent analysis web pages according to claim 1, characterized in that: The steps to obtain the video stream collected by the target camera include: Obtain identification information of the target camera, and match the camera RTSP address of the target camera in the camera address list based on the identification information; The camera RTSP address is subjected to an availability test, and after the availability test is passed, the camera RTSP address is used to pull the video stream of the target monitoring area collected by the target camera.

3. The method for real-time visualization of intelligent analysis web pages according to claim 1, characterized in that: The step of performing intelligent analysis on each video frame of the video stream to obtain a real-time analysis result of each video frame in the video stream specifically includes: The video streams are respectively input into a plurality of intelligent analysis models corresponding to different analysis contents for the target monitoring area, and real-time analysis contents of each video frame output by the plurality of intelligent analysis models are obtained; Based on the similarity matching results of the analysis content of each intelligent analysis model and the monitoring content of each visual monitoring terminal, the association relationship between the real-time analysis content output by the intelligent analysis model and the visual monitoring terminal is determined, and real-time analysis content mapping information consisting of a plurality of real-time analysis contents with association relationship and visual monitoring terminals is established; According to the real-time analysis content mapping information corresponding to each intelligent analysis model, a real-time analysis result of each video frame in the video stream is generated.

4. The method for real-time visualization of intelligent analysis web pages according to claim 1, characterized in that: According to the hardware configuration status information and application analysis content of several visual monitoring terminals, the visual monitoring terminals are divided into high-synchronization visual monitoring terminals and low-synchronization visual monitoring terminals, specifically including: Acquire hardware configuration status information of a plurality of visual monitoring terminals, and divide the visual monitoring terminals into high-synchronization visual monitoring terminals and low-synchronization visual monitoring terminals according to the hardware configuration status information and preset hardware configuration status requirements required for performing text and video synchronization; Extracting text information of real-time analysis content corresponding to each high-synchronous visual monitoring terminal, and calculating, based on the number of text display words and the text display format in the text information, a quantized value of encoding performance consumption when encoding and a quantized value of decoding performance consumption when decoding the real-time analysis content corresponding to each high-synchronous visual monitoring terminal according to a preset conversion relationship; The first constraint condition is that the sum of the quantized values ​​of the encoding performance consumption of the real-time analysis content corresponding to the selected target high-synchronous visual monitoring terminals in the previous monitoring cycle is less than the quantized value of the encoding standard performance consumption of the encoding server, and the second constraint condition is that the sum of the quantized values ​​of the decoding performance consumption of the real-time analysis content corresponding to the selected target high-synchronous visual monitoring terminals in the previous monitoring cycle is less than the quantized value of the decoding standard performance consumption of each visual monitoring terminal; the optimization goal is to maximize the number of selected target high-synchronous visual monitoring terminals, optimize the selection strategy of the target high-synchronous visual monitoring terminals in the current monitoring cycle, and determine the target high-synchronous visual monitoring terminals.

5. The method for real-time visualization of intelligent analysis web pages according to claim 1, characterized in that: When a video stream pulling instruction is received, the video stream data obtained by transcoding is pulled at the visual monitoring terminal, and the video stream and real-time analysis content steps in the video stream data are extracted, specifically including: When a video stream pull instruction is received, the FLV video stream data obtained by transcoding is pulled at the visual monitoring terminal; wherein the video stream data includes real-time analysis content of a non-target high-synchronous visual monitoring terminal and video stream encoding data with real-time analysis content corresponding to a target high-synchronous visual monitoring terminal and a low-synchronous visual monitoring terminal; The real-time analysis content of the non-target high-synchronous visual monitoring terminal and the video stream encoding data are extracted from the video stream data, the video stream encoding data is used as the video generation data, and the real-time analysis content of the non-target high-synchronous visual monitoring terminal is used as the video rendering data.

6. The method for real-time visualization of intelligent analysis web pages according to claim 5, characterized in that: The step of generating a playback video based on the video stream, drawing the analysis result on a video frame corresponding to a timestamp in the playback video, and obtaining a video analysis visualization result specifically includes: The video generation data is sent to a decoder of a visualization monitoring terminal, and the decoder is used to generate a decoded video containing real-time analysis content corresponding to the original video and the target high-synchronization visualization monitoring terminal and the low-synchronization visualization monitoring terminal; Based on the video rendering data, the real-time analysis content of the non-target high-synchronization visualization monitoring terminal is drawn on the decoded video to obtain a video analysis visualization result.

7. An intelligent analysis web page real-time visualization device, characterized in that: include: An acquisition module is used to acquire a video stream collected by a target camera, perform intelligent analysis on each video frame of the video stream, and obtain a real-time analysis result of each video frame in the video stream; wherein the real-time analysis result includes real-time analysis contents corresponding to a plurality of visual monitoring terminals respectively; The encapsulation module is used to encapsulate the video stream and several real-time analysis contents of each video frame according to the real-time analysis result, and send the encapsulated video analysis stream to the streaming media server, so that the streaming media server transcodes the video analysis stream into the target type of video stream data; specifically comprising: dividing the visual monitoring terminals into high-synchronicity visual monitoring terminals and low-synchronicity visual monitoring terminals according to the hardware configuration status information and application analysis contents of several visual monitoring terminals; wherein the high-synchronicity visual monitoring terminals include target high-synchronicity visual monitoring terminals and non-target high-synchronicity visual monitoring terminals; encoding the real-time analysis contents corresponding to the target high-synchronicity visual monitoring terminals and the low-synchronicity visual monitoring terminals and each video frame of the video stream, and encapsulating the encoded video stream and several real-time analysis contents of the non-target high-synchronicity visual monitoring terminals into NAL frames using JSON, merging the NAL frames into ES code streams for packaging and MPEGES encapsulation; using RTP to send the video analysis stream obtained by MPEGES encapsulation to the streaming media server, so that the streaming media server transcodes the video analysis stream into FLV video stream data; An extraction module, for pulling the video stream data obtained by transcoding at a visual monitoring terminal when receiving a video stream pulling instruction, and extracting the video stream and real-time analysis content from the video stream data; A generation module is used to generate a playback video based on the video stream, draw the analysis result on a video frame corresponding to a timestamp in the playback video, and obtain a video analysis visualization result.

8. An intelligent analysis web page real-time visualization device, characterized in that: The intelligent analysis web page real-time visualization device includes: a memory, a processor, and an intelligent analysis web page real-time visualization program stored in the memory and executable on the processor. When the intelligent analysis web page real-time visualization program is executed by the processor, the steps of the intelligent analysis web page real-time visualization method as described in any one of claims 1 to 6 are implemented.

9. A storage medium, characterized in that: The storage medium stores an intelligent analysis web page real-time visualization program, which, when executed by a processor, implements the steps of the intelligent analysis web page real-time visualization method according to any one of claims 1 to 6.

Images