Video data acquisition method and device of monitoring terminal

Abstract

The invention discloses a video data acquisition method of a monitoring terminal, and relates to the technical field of video data processing. The method comprises the steps that when it is judged that no recognition target exists in initial video data, a target picture in the initial video data is acquired, the target picture is recognized based on an online monitoring recognition model, and a recognition result of the online monitoring recognition model is acquired; and if the recognition result meets a preset model upgrading condition, executing an upgrading operation on the online monitoring recognition model according to the target picture to obtain an upgraded online monitoring recognition model. According to the invention, the monitoring terminal is tightly matched with the online monitoring identification model, real-time intelligent identification of monitoring data can be realized, and the online monitoring identification model can upgrade and update itself according to the identification condition of the current video, so that the online monitoring identification model can well identify the latest monitoring background picture, and the monitoring efficiency is improved. Therefore, the accuracy of intelligent identification of the video data is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and in particular to a method and device for acquiring video data of a monitoring terminal. Background Art

[0002] The intelligent monitoring terminal obtained by combining the monitoring terminal with the monitoring recognition model can realize intelligent monitoring, for example, intelligent recognition of target objects (vehicles or pedestrians) in road monitoring, and even intelligent reporting of monitoring events. Unlike the existing intelligent monitoring system, the intelligent monitoring terminal itself has intelligence, and the monitoring data can be intelligently processed inside the monitoring terminal, and then the data sent carries the results of intelligent recognition, which improves the efficiency of intelligent monitoring. Therefore, the intelligent monitoring terminal can improve the efficiency of intelligent monitoring in road monitoring or IoT monitoring.

[0003] However, loading the monitoring recognition model in the monitoring terminal will also increase the cost of the monitoring terminal. Therefore, the volume (memory usage and computing power) of the monitoring recognition model loaded into the monitoring terminal is generally not large. This is because the location of the monitoring terminal is fixed or the usage scenario is fixed, such as a monitoring system fixed on a road gantry, a home smart monitoring system fixed at a certain location in the home, etc. At this time, it is only necessary to allow the monitoring system to recognize the background of the monitoring screen, so that the target can be quickly identified from the background. Under this method, the monitoring recognition model algorithm is very simple and its volume is also small. Correspondingly, the intelligent recognition efficiency is also higher. On this basis, if you want to obtain more accurate or customized intelligent recognition results, you can further process the video data.

[0004] In the above context, the monitoring terminal should have a sensitive perception of the environment in which it is located. Therefore, the monitoring recognition model needs to be able to accurately identify the background of the monitoring environment. However, the background often changes over time. If the monitoring recognition model is not updated in time, the recognition accuracy will be reduced. Therefore, it is necessary to propose a new video data acquisition method for monitoring terminals to improve the accuracy of intelligent recognition of video data. Summary of the invention

[0005] The present invention provides a method and device for acquiring video data of a monitoring terminal, which are used to improve the accuracy of intelligent recognition of video data.

[0006] In order to solve the above technical problems, the first aspect of the present invention discloses a method for acquiring video data of a monitoring terminal, the method comprising: Acquire initial video data captured by the monitoring terminal, and determine whether there is an identification target in the initial video data according to the initial video data; When it is determined that the identification target does not exist in the initial video data, obtaining a target image in the initial video data, identifying the target image based on an online monitoring identification model, and obtaining an identification result of the online monitoring identification model; Determine whether the recognition result meets the preset model upgrade condition. If the recognition result meets the preset model upgrade condition, perform an upgrade operation on the online monitoring recognition model according to the target image to obtain an upgraded online monitoring recognition model; The upgraded online monitoring recognition model is used to perform intelligent recognition operations on the initial video data captured by the monitoring terminal to obtain target video data.

[0007] As an optional implementation manner, in the first aspect of the present invention, the identifying the target image based on the online monitoring recognition model and obtaining the recognition result of the online monitoring recognition model includes: Dividing the target image into multiple sub-areas according to a preset division strategy; Inserting a preset pattern to be identified in each sub-region, for each sub-region, determining a pixel region within a preset range around an edge position of the pattern to be identified as a transition region, performing a Gaussian filter operation on the transition region, and obtaining a target processing image; Identify the target processing image based on the online monitoring recognition model, and obtain a pattern recognition result of the online monitoring recognition model, wherein the pattern recognition result includes a pattern recognized by the online monitoring recognition model from the target processing image; And, judging whether the recognition result meets the preset model upgrade condition includes: Comparing the pattern recognition result with the pattern to be recognized, and calculating the recognition accuracy of the pattern recognition result; If the accuracy of the pattern recognition result is lower than a preset accuracy threshold, it is determined that the recognition result meets a preset model upgrade condition.

[0008] As an optional implementation manner, in the first aspect of the present invention, dividing the target image into a plurality of sub-regions according to a preset division strategy includes: Determining the distances from a plurality of reference points preset in the target image to the monitoring terminal according to the lens parameters corresponding to the monitoring terminal and the target image; Divide all reference points into a plurality of sub-reference point sets, wherein, in each of the sub-reference point sets, the distances from all the sub-reference points to the monitoring terminal are within a preset distance range corresponding to the sub-reference point set; According to all the sub-reference point sets, the target image is divided into a plurality of sub-regions, wherein each of the sub-reference point sets corresponds to a sub-region.

[0009] As an optional implementation manner, in the first aspect of the present invention, performing a Gaussian filtering operation on the transition region to obtain a target processed image includes: Determining Gaussian filter algorithm parameters according to the preset distance range corresponding to each sub-area; A Gaussian filtering operation is performed on the transition area according to Gaussian filtering algorithm parameters corresponding to the transition area to obtain a target processed image.

[0010] As an optional implementation, in the first aspect of the present invention, performing intelligent recognition operation on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain target video data includes: Using the upgraded online monitoring recognition model, an intelligent recognition operation is performed on the initial video data captured by the monitoring terminal to obtain an intelligent recognition result; Based on the intelligent recognition result, extracting a target character from a static character library loaded in advance, wherein the target character is used to represent the intelligent recognition result in a preset expression; The target character is superimposed on the video stream corresponding to the initial video data through a rendering operation to obtain target video data.

[0011] As an optional embodiment, in the first aspect of the present invention, the method further comprises: Pulling a target video stream corresponding to the target video data from the monitoring terminal, decoding the target video stream according to a preset decoding scheme, and obtaining a decoded target video stream; According to the decoded target video stream, multiple derivative video streams corresponding to the target video stream are obtained according to a preset data multiplexing scheme, and according to a coding scheme corresponding to the preset decoding scheme, a re-encoding operation is performed on each derivative video stream to obtain multiple derivative video data.

[0012] As an optional implementation, in the first aspect of the present invention, performing intelligent recognition operation on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain target video data includes: Using the upgraded online monitoring recognition model, an intelligent recognition operation is performed on the initial video data captured by the monitoring terminal to obtain an intelligent recognition result; Based on the intelligent recognition result, extracting a derived target character corresponding to each channel of derived video data from a pre-loaded static character library, wherein the derived target character corresponds to the resolution and bit rate of the corresponding derived video data, and the derived target character is used to represent the intelligent recognition result in a preset expression; For each channel of the derived video data, the derived target characters are superimposed on the video stream corresponding to the derived video data through a rendering operation to obtain target derived video data.

[0013] As an optional embodiment, in the first aspect of the present invention, the method further comprises: Acquire original audio data recorded when the monitoring terminal captures initial video data, extract audio features from the original audio data and generate an audio feature set; Determining whether there is a matching audio feature corresponding to a predetermined recognition target in the audio feature set; And, judging whether there is an identification target in the initial video data according to the initial video data, comprising: The initial video data is identified based on an online monitoring recognition model. If the online monitoring recognition model identifies a recognition target from the initial video data, or determines that the matching audio feature exists in the audio feature set, it is determined that the recognition target exists in the initial video data.

[0014] A second aspect of the present invention discloses a video data acquisition device for a monitoring terminal, the device comprising: The target recognition module is used to obtain the initial video data captured by the monitoring terminal, and determine whether there is an identification target in the initial video data according to the initial video data; A model testing module, for obtaining a target image in the initial video data when it is determined that the identification target does not exist in the initial video data, identifying the target image based on the online monitoring identification model, and obtaining an identification result of the online monitoring identification model; A model upgrade module, used to determine whether the recognition result meets the preset model upgrade condition, and if the recognition result meets the preset model upgrade condition, perform an upgrade operation on the online monitoring recognition model according to the target image to obtain an upgraded online monitoring recognition model; The intelligent recognition module is used to perform intelligent recognition operations on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain target video data.

[0015] As an optional implementation, in the second aspect of the present invention, the model testing module identifies the target image based on the online monitoring recognition model, and the specific operation method of obtaining the recognition result of the online monitoring recognition model includes: Dividing the target image into multiple sub-areas according to a preset division strategy; Inserting a preset pattern to be identified in each sub-region, for each sub-region, determining a pixel region within a preset range around an edge position of the pattern to be identified as a transition region, performing a Gaussian filter operation on the transition region, and obtaining a target processing image; Identify the target processing image based on the online monitoring recognition model, and obtain a pattern recognition result of the online monitoring recognition model, wherein the pattern recognition result includes a pattern recognized by the online monitoring recognition model from the target processing image; And, the specific operation method of the model upgrade module to determine whether the recognition result meets the preset model upgrade condition includes: Comparing the pattern recognition result with the pattern to be recognized, and calculating the recognition accuracy of the pattern recognition result; If the accuracy of the pattern recognition result is lower than a preset accuracy threshold, it is determined that the recognition result meets a preset model upgrade condition.

[0016] As an optional implementation, in the second aspect of the present invention, the specific operation of the model testing module dividing the target image into a plurality of sub-areas according to a preset division strategy includes: Determining the distances from a plurality of reference points preset in the target image to the monitoring terminal according to the lens parameters corresponding to the monitoring terminal and the target image; Divide all reference points into a plurality of sub-reference point sets, wherein, in each of the sub-reference point sets, the distances from all the sub-reference points to the monitoring terminal are within a preset distance range corresponding to the sub-reference point set; According to all the sub-reference point sets, the target image is divided into a plurality of sub-regions, wherein each of the sub-reference point sets corresponds to a sub-region.

[0017] As an optional implementation, in the second aspect of the present invention, the model testing module performs a Gaussian filtering operation on the transition region to obtain a specific operation mode of the target processing image, including: Determining Gaussian filter algorithm parameters according to the preset distance range corresponding to each sub-area; A Gaussian filtering operation is performed on the transition area according to Gaussian filtering algorithm parameters corresponding to the transition area to obtain a target processed image.

[0018] As an optional implementation, in the second aspect of the present invention, the intelligent recognition module performs an intelligent recognition operation on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain the specific operation mode of the target video data, including: Using the upgraded online monitoring recognition model, an intelligent recognition operation is performed on the initial video data captured by the monitoring terminal to obtain an intelligent recognition result; Based on the intelligent recognition result, extracting a target character from a static character library loaded in advance, wherein the target character is used to represent the intelligent recognition result in a preset expression; The target character is superimposed on the video stream corresponding to the initial video data through a rendering operation to obtain target video data.

[0019] As an optional embodiment, in the second aspect of the present invention, the device further includes: A video decoding module, used to pull a target video stream corresponding to the target video data from the monitoring terminal, decode the target video stream according to a preset decoding scheme, and obtain a decoded target video stream; The data derivation module is used to obtain multiple derived video streams corresponding to the target video stream according to the decoded target video stream according to a preset data multiplexing scheme, and to perform a re-encoding operation on each derived video stream according to a coding scheme corresponding to the preset decoding scheme to obtain multiple derived video data.

[0020] As an optional implementation, in the second aspect of the present invention, the intelligent recognition module performs an intelligent recognition operation on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain the specific operation mode of the target video data, including: Using the upgraded online monitoring recognition model, an intelligent recognition operation is performed on the initial video data captured by the monitoring terminal to obtain an intelligent recognition result; Based on the intelligent recognition result, extracting a derived target character corresponding to each channel of derived video data from a pre-loaded static character library, wherein the derived target character corresponds to the resolution and bit rate of the corresponding derived video data, and the derived target character is used to represent the intelligent recognition result in a preset expression; For each channel of the derived video data, the derived target characters are superimposed on the video stream corresponding to the derived video data through a rendering operation to obtain target derived video data.

[0021] As an optional embodiment, in the second aspect of the present invention, the device further includes: The audio analysis module is used to obtain the original audio data recorded when the monitoring terminal captures the initial video data, extract the audio features in the original audio data and generate an audio feature set; determine whether there is a matching audio feature corresponding to a predetermined recognition target in the audio feature set; Furthermore, the target recognition module determines, based on the initial video data, whether there is a specific operation mode for identifying the target in the initial video data, including: The initial video data is identified based on an online monitoring recognition model. If the online monitoring recognition model identifies a recognition target from the initial video data, or determines that the matching audio feature exists in the audio feature set, it is determined that the recognition target exists in the initial video data.

[0022] The third aspect of the present invention discloses another video data acquisition system for a monitoring terminal, the system comprising: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the video data acquisition method of the monitoring terminal disclosed in the first aspect of the present invention.

[0023] Compared with the prior art, the monitoring terminal in the present invention works closely with the online monitoring recognition model to realize real-time intelligent recognition of monitoring data, and the online monitoring recognition model can upgrade and update itself according to the recognition situation of the current video, so that the online monitoring recognition model can well recognize the latest monitoring background picture, thereby improving the accuracy of intelligent recognition of video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a flow chart of a method for acquiring video data of a monitoring terminal disclosed in an embodiment of the present invention; Figure 2 It is a structural schematic diagram of a video data acquisition device of a monitoring terminal disclosed in an embodiment of the present invention; Figure 3 The present invention is a schematic diagram of the structure of a video data acquisition system for a monitoring terminal disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0028] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0029] The invention discloses a method and a device for acquiring video data of a monitoring terminal, which are used to improve the accuracy of intelligent recognition of video data.

[0030] Embodiment 1 See also Figure 1 , Figure 1 1 is a flow chart of a method for acquiring video data of a monitoring terminal disclosed in an embodiment of the present invention. Figure 1 The video data acquisition method of the monitoring terminal described above can be integrated into a video data acquisition device of the monitoring terminal or in the monitoring terminal. The video data acquisition device of the monitoring terminal can be integrated into a cloud server or a local server. Figure 1 As shown, the video data acquisition method of the monitoring terminal may include the following operations: Step 101: Acquire initial video data captured by a monitoring terminal, and determine whether there is an identification target in the initial video data based on the initial video data.

[0031] In the embodiment of the present invention, the monitoring terminal cooperates with the online monitoring recognition model to realize intelligent processing of video data, wherein the online monitoring recognition model can be integrated in the monitoring terminal or in other devices connected to the monitoring terminal. Based on the cooperation between the monitoring terminal and the online monitoring recognition model, it is possible to judge whether there is an identification target in the initial video data according to the initial video data.

[0032] Step 102: When it is determined that the identification target does not exist in the initial video data, a target image in the initial video data is obtained, the target image is identified based on the online monitoring identification model, and the identification result of the online monitoring identification model is obtained.

[0033] In the embodiment of the present invention, when the initial video data has a recognition target, the online monitoring recognition model needs to perform the recognition operation in real time, and because the recognition target exists in the picture and the complete background picture cannot be obtained, the online monitoring recognition model is not upgraded at this time. If the initial video data does not have a recognition target, the picture of the video data at this time is the latest background picture, and further model upgrade operations need to be performed so that the latest model can recognize the latest background picture.

[0034] Specifically, the target image is identified based on the online monitoring recognition model, and the recognition result of the online monitoring recognition model is obtained. The recognition result here can be the recognition result obtained after the online monitoring recognition model identifies the target image according to the normal processing flow; the online monitoring recognition model can also be set to a test mode. The online recognition model in the test mode can identify the target image and extract the features of each element in the target image.

[0035] In the embodiment of the present invention, optionally, the online monitoring recognition model can recognize the background of the monitoring screen, and then quickly recognize the target from the background. At this time, the monitoring recognition model algorithm is simple and the intelligent recognition efficiency is high.

[0036] Step 103: determine whether the recognition result meets the preset model upgrade conditions. If the recognition result meets the preset model upgrade conditions, perform an upgrade operation on the online monitoring recognition model according to the target image to obtain an upgraded online monitoring recognition model.

[0037] In the embodiment of the present invention, it is necessary to further determine whether the recognition result indicates that the current online monitoring recognition model needs to be upgraded, that is, to determine whether the current online monitoring recognition model recognizes the current target image. Specifically, the embodiment of the present invention determines whether the recognition result meets the preset model upgrade conditions. If the recognition result meets the preset model upgrade conditions, it means that the current online monitoring recognition model needs to be upgraded. Then, the online monitoring recognition model is upgraded according to the target image, so that the online monitoring recognition model can recognize the current target image as the background, thereby obtaining the upgraded online monitoring recognition model.

[0038] Step 104: Using the upgraded online monitoring recognition model, perform intelligent recognition operations on the initial video data captured by the monitoring terminal to obtain target video data.

[0039] In an embodiment of the present invention, after the monitoring terminal ingests the initial video data, the upgraded online monitoring recognition model performs intelligent recognition operations on the initial video data, such as identifying the recognition target in the video data, determining the status of the recognition target in the video data, issuing an early warning based on the video data, and determining whether the video data meets the execution conditions of certain operations. Combined with the intelligent recognition operation, the data finally obtained is the target video data.

[0040] It can be seen that in the embodiment of the present invention, the monitoring terminal and the online monitoring recognition model work closely together to realize real-time intelligent recognition of monitoring data, and the online monitoring recognition model can upgrade and update itself according to the recognition situation of the current video, so that the online monitoring recognition model can well recognize the latest monitoring background picture, thereby improving the accuracy of intelligent recognition of video data.

[0041] In an optional embodiment, recognizing the target image based on the online monitoring recognition model and obtaining the recognition result of the online monitoring recognition model may include: The target image is divided into multiple sub-areas according to a preset division strategy. The recognition ability of the target image should be the recognition ability of the target image as a whole. Therefore, the target image can be divided into multiple sub-areas, and then each sub-area is recognized separately, so as to comprehensively judge the recognition ability of the online monitoring recognition model.

[0042] A preset pattern to be identified is inserted in each sub-region. In this optional embodiment, the core principle of judging the recognition ability of the online monitoring recognition model is: inserting a preset pattern to be identified in each sub-region. Optionally, the pattern to be identified can be a specific graphic, shape, non-vector symbol, or a pattern of an identification target such as a vehicle, pedestrian, or specific object. In which, the pattern to be identified is directly inserted into the sub-region. Because there is a difference between the pixels of the pattern to be identified and the sub-region, it is often easy to identify the pattern to be identified from the sub-region. In order to make the insertion of the pattern to be identified appear more natural and closer to the image obtained in the formal real scene, in this optional embodiment, Gaussian filtering is used to process the connection between the pattern to be identified and the sub-region.

[0043] Specifically, for each sub-region, the pixel area within the preset range around the edge position of the pattern to be identified is determined as the transition area, and the Gaussian filtering operation is performed on the transition area to obtain the target processing image. Optionally, in order to improve the operation efficiency, the preset range is selected as N pixels on the left and N pixels on the right of the edge pixels of the pattern to be identified, where N is the preset number of pixels, so as to facilitate computer operation. In this embodiment, the Gaussian filtering method is used to process the connection between the pattern to be identified and the sub-region, so that the fusion of the pattern to be identified and the sub-region is more natural and real, and the reliability of subsequent test results is improved.

[0044] Based on the online monitoring recognition model, the target processing image is identified, and the pattern recognition result of the online monitoring recognition model is obtained, wherein the pattern recognition result may include the pattern recognized by the online monitoring recognition model from the target processing image, that is, the pattern area to be recognized recognized by the online monitoring recognition model from the target processing image.

[0045] Furthermore, the above-mentioned determination of whether the recognition result meets the preset model upgrade condition may include: Compare the pattern recognition result with the pattern to be recognized, and calculate the recognition accuracy of the pattern recognition result; for example, for each pattern to be recognized, use the pattern similarity analysis method to determine the similarity between the pattern recognition result and the pattern to be recognized. When the similarity is lower than a certain threshold, it is determined to be a recognition error. The proportion of correctly recognized patterns among all the patterns to be recognized is the recognition accuracy.

[0046] If the accuracy of the pattern recognition result is lower than a preset accuracy threshold, it is determined that the recognition result meets the preset model upgrade condition.

[0047] In this optional embodiment, the target image is divided into multiple sub-areas according to a preset division strategy, and the connection between the pattern to be identified and the sub-area is processed by Gaussian filtering, so that the fusion of the pattern to be identified and the sub-area is more natural and realistic, thereby improving the reliability of the recognition result in satisfying the preset model upgrade condition judgment process.

[0048] In another optional embodiment, for the monitoring terminal, without installing a special lens, the acquired video and image data generally meet the conditions that the near is large and the far is small, the near is clear and the far is blurred. Therefore, if each sub-area is subject to the same recognition requirements, it is inconsistent with the actual situation. For this reason, in this optional embodiment, the above-mentioned dividing the target image into multiple sub-areas according to the preset division strategy may include: According to the lens parameters and target image corresponding to the monitoring terminal, the distances from multiple reference points preset in the target image to the monitoring terminal are determined; the lens parameters include a series of parameters such as distortion coefficient and focal length. According to the lens parameters and target image, the image coordinates of each point on the target image can be converted into world coordinates, and the actual position of each point can be calculated.

[0049] All reference points are divided into multiple sub-reference point sets, wherein, in each sub-reference point set, the distances from all sub-reference points to the monitoring terminal are within the preset distance range corresponding to the sub-reference point set. Through the above operation, sub-reference points that are close in distance and clustered together are grouped into the same sub-reference point set, and then the target image is divided into multiple sub-areas based on all sub-reference point sets, wherein each sub-reference point set corresponds to a sub-area. In this way, on the entire target image, some positions that are closer to the lens and some positions that are farther from the lens will appear in different sub-areas.

[0050] It can be seen that this optional embodiment divides different sub-areas according to the distance between them and the camera lens of the monitoring terminal, so that the actual positions of images in the same sub-area are close, thereby dividing the sub-areas more reasonably.

[0051] In this optional embodiment, further optionally, the above-mentioned performing a Gaussian filtering operation on the transition area to obtain a target processed image may include: According to the preset distance range corresponding to each sub-region, the Gaussian filter algorithm parameters are determined; specifically, the Gaussian filter algorithm parameters generally include the filter kernel size (ksize) and the Gaussian filter standard deviation (σ), where the filter kernel size refers to the width and height of the convolution kernel, which is usually an odd number (such as 3×3, 5×5, etc.). It determines the range of neighborhood pixels during the filtering process. The larger the filter kernel, the more obvious the smoothing effect, but the computational complexity will also increase. The standard deviation (σ) is a key parameter of the Gaussian function and determines the degree of diffusion of the Gaussian distribution. The larger σ is, the flatter the weight distribution of the Gaussian kernel is and the stronger the smoothing effect is; the smaller σ is, the weight is concentrated in the center and the smoothing effect is weaker. Generally, for sub-regions with a farther preset distance range, the set filter kernel and standard deviation are smaller, and for sub-regions with a closer preset distance range, the set filter kernel and standard deviation are larger. As a result, the closer the sub-region is, the more natural the combination of the pattern to be identified and the sub-region is, and the higher the recognition capability requirement of the online monitoring recognition model is; the farther the sub-region is, the lower the naturalness requirement of the combination of the pattern to be identified and the sub-region is, and the lower the recognition capability requirement of the online monitoring recognition model is.

[0052] According to the Gaussian filter algorithm parameters corresponding to the transition area, a Gaussian filter operation is performed on the transition area to obtain a target processing image. In the target processing image obtained by the above method, different sub-areas correspond to different Gaussian filter algorithms according to their distances, so that the obtained target processing image is closer to the actual situation and the reliability of the subsequent judgment process is improved.

[0053] In another optional embodiment, the results recognized by the online monitoring recognition model are generally described in computer language, but in some scenarios, users need to be able to see the description of the results, such as inserting corresponding characters or agreed symbols in the video screen, so that users can see the recognition results in real time. In the prior art, the traditional OSD character overlay technology process is relatively complicated. It is necessary to first generate the characters into a picture format, then decode the picture, and finally overlay the decoded picture with the decoded data of the target video. The whole process is extremely time-consuming and seriously affects the operating efficiency of the system.

[0054] Therefore, in this optional embodiment, the above-mentioned intelligent recognition operation is performed on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain the target video data, which may include: Through the upgraded online monitoring recognition model, the initial video data captured by the monitoring terminal is intelligently recognized to obtain intelligent recognition results; Based on the intelligent recognition result, a target character is extracted from a pre-loaded static character library, wherein the target character is used to represent the intelligent recognition result in a preset expression mode; The target character is superimposed on the video stream corresponding to the initial video data through a rendering operation to obtain the target video data.

[0055] In this optional embodiment, a technical solution combining static loading and dynamic mapping is adopted. For example, a complete set of static character libraries can be generated in advance and loaded into the memory in advance. When superimposing characters, the character generation step can be directly skipped, and only the rendering operation is required to complete the character superposition, which greatly improves the efficiency of character superposition.

[0056] In another optional embodiment, the data sent by the monitoring terminal is often sent to different host computers, and different host computers may require different resolutions and bit rates. In the traditional solution, the above tasks are handed over to the downstream network for execution. When the number of monitoring terminals is large, it will bring great pressure to the downstream network bandwidth. Therefore, in this optional embodiment, the method may also include: Pull a target video stream corresponding to a target video data from the monitoring terminal, decode the target video stream according to a preset decoding scheme, and obtain a decoded target video stream; According to the decoded target video stream, multiple derivative video streams corresponding to the target video stream are obtained according to a preset data multiplexing scheme, and a re-encoding operation is performed on each derivative video stream according to a coding scheme corresponding to the preset decoding scheme to obtain multiple derivative video data.

[0057] The above scheme is illustrated as follows: In the video compression and transcoding technology system, the RTSP protocol can be used. With its excellent compatibility, the protocol can achieve seamless connection with cameras of various brands and manufacturers. In the actual operation process, the software only needs to pull one high-definition video stream from the monitoring terminal and decode the H264 data in it once. Subsequently, with the help of data multiplexing technology and re-encoding technology, it can efficiently output multiple standard H264 video streams with different resolutions and bit rates. This solution not only fully meets the complex and diverse business needs, but also significantly reduces the transmission pressure of the downstream network bandwidth, providing a strong guarantee for the stable operation of the system.

[0058] In this optional embodiment, further optionally, in the case of a multiplexed scenario, the above-mentioned upgraded online monitoring recognition model is used to perform intelligent recognition operations on the initial video data captured by the monitoring terminal to obtain the target video data, which may also include: Through the upgraded online monitoring recognition model, the initial video data captured by the monitoring terminal is intelligently recognized to obtain intelligent recognition results; Based on the intelligent recognition result, the derived target characters corresponding to each channel of derived video data are extracted from the pre-loaded static character library, wherein the derived target characters correspond to the resolution and bit rate of the corresponding derived video data, and the derived target characters are used to represent the intelligent recognition result in a preset expression mode; For each channel of derived video data, the derived target characters are superimposed on the video stream corresponding to the derived video data through a rendering operation to obtain the target derived video data.

[0059] In this optional embodiment, for video streams with different resolutions and bit rates, character information adapted thereto can be flexibly superimposed, thereby achieving an organic combination of static loading and dynamic mapping and improving the flexibility of character superposition.

[0060] In another optional embodiment, the identification target is often accompanied by its unique sound, such as human footsteps, vehicle engine sound, etc. In many cases, the identification target has not appeared in the video screen, but the sound it emits can be detected. At this time, it often means that the identification target is nearby and may appear in the video screen at any time. Therefore, in order to improve the identification reliability of the identification target, the method may also include: Obtaining the original audio data recorded when the monitoring terminal captures the initial video data, extracting the audio features in the original audio data and generating an audio feature set; Determine whether there is a matching audio feature corresponding to a predetermined recognition target in the audio feature set; wherein the matching audio feature corresponding to the predetermined recognition target can be a specific footstep sound, a door opening sound, and any sound made by a human being, or it can be a vehicle engine sound, a vehicle horn sound, etc.

[0061] And, judging whether there is a recognition target in the initial video data according to the initial video data may include: The initial video data is identified based on the online monitoring recognition model. If the online monitoring recognition model identifies a recognition target from the initial video data, or determines that there is a matching audio feature in the audio feature set, it is determined that the recognition target exists in the initial video data.

[0062] In this optional embodiment, the determination of whether the identification target exists is comprehensively determined by combining the video image and sound feature information, thereby improving the reliability of the determination of whether the identification target exists.

[0063] Embodiment 2 See also Figure 2 , Figure 2 Schematic diagram of the structure of a video data acquisition device for a monitoring terminal disclosed in an embodiment of the present invention. Figure 2 As shown, the video data acquisition device of the monitoring terminal may include: The target recognition module 201 is used to obtain the initial video data captured by the monitoring terminal, and determine whether there is an identification target in the initial video data according to the initial video data; The model testing module 202 is used to obtain a target image in the initial video data when it is determined that the target does not exist in the initial video data, identify the target image based on the online monitoring recognition model, and obtain a recognition result of the online monitoring recognition model; The model upgrade module 203 is used to determine whether the recognition result meets the preset model upgrade condition. If the recognition result meets the preset model upgrade condition, the online monitoring recognition model is upgraded according to the target image to obtain the upgraded online monitoring recognition model. The intelligent recognition module 204 is used to perform intelligent recognition operations on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain target video data.

[0064] In an optional embodiment, the specific operation of the model testing module 202 identifying the target image based on the online monitoring recognition model and obtaining the recognition result of the online monitoring recognition model may include: Divide the target image into multiple sub-areas according to a preset division strategy; Inserting a preset pattern to be identified in each sub-region, for each sub-region, determining a pixel region within a preset range around an edge position of the pattern to be identified as a transition region, performing a Gaussian filter operation on the transition region, and obtaining a target processed image; Recognize the target processing image based on the online monitoring recognition model, and obtain a pattern recognition result of the online monitoring recognition model, wherein the pattern recognition result may include a pattern recognized by the online monitoring recognition model from the target processing image; Furthermore, the specific operation of the model upgrade module 203 to determine whether the recognition result meets the preset model upgrade condition may include: Compare the pattern recognition result with the pattern to be recognized, and calculate the recognition accuracy of the pattern recognition result; If the accuracy of the pattern recognition result is lower than a preset accuracy threshold, it is determined that the recognition result meets the preset model upgrade condition.

[0065] In another optional embodiment, the specific operation of the model testing module 202 dividing the target image into multiple sub-areas according to the preset division strategy may include: According to the lens parameters corresponding to the monitoring terminal and the target image, determine the distances from multiple reference points preset in the target image to the monitoring terminal; Divide all reference points into multiple sub-reference point sets, wherein, in each sub-reference point set, the distances from all sub-reference points to the monitoring terminal are within a preset distance range corresponding to the sub-reference point set; According to all sub-reference point sets, the target image is divided into multiple sub-regions, wherein each sub-reference point set corresponds to a sub-region.

[0066] In yet another optional embodiment, the model testing module 202 performs a Gaussian filtering operation on the transition region to obtain a specific operation mode of the target processing image, which may include: Determine Gaussian filter algorithm parameters according to a preset distance range corresponding to each sub-area; A Gaussian filtering operation is performed on the transition area according to the Gaussian filtering algorithm parameters corresponding to the transition area to obtain the target processed image.

[0067] In another optional embodiment, the intelligent recognition module 204 performs intelligent recognition operations on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain the target video data. The specific operation method may include: Through the upgraded online monitoring recognition model, the initial video data captured by the monitoring terminal is intelligently recognized to obtain intelligent recognition results; Based on the intelligent recognition result, a target character is extracted from a pre-loaded static character library, wherein the target character is used to represent the intelligent recognition result in a preset expression mode; The target character is superimposed on the video stream corresponding to the initial video data through a rendering operation to obtain the target video data.

[0068] In yet another optional embodiment, the device may further include: The video decoding module is used to pull a target video stream corresponding to a target video data from the monitoring terminal, decode the target video stream according to a preset decoding scheme, and obtain the decoded target video stream; The data derivation module is used to obtain multiple derived video streams corresponding to the target video stream according to the decoded target video stream according to a preset data multiplexing scheme, and to perform a re-encoding operation on each derived video stream according to a coding scheme corresponding to the preset decoding scheme to obtain multiple derived video data.

[0069] In another optional embodiment, the intelligent recognition module 204 performs intelligent recognition operations on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain the target video data. The specific operation method may include: Through the upgraded online monitoring recognition model, the initial video data captured by the monitoring terminal is intelligently recognized to obtain intelligent recognition results; Based on the intelligent recognition result, the derived target characters corresponding to each channel of derived video data are extracted from the pre-loaded static character library, wherein the derived target characters correspond to the resolution and bit rate of the corresponding derived video data, and the derived target characters are used to represent the intelligent recognition result in a preset expression mode; For each channel of derived video data, the derived target characters are superimposed on the video stream corresponding to the derived video data through a rendering operation to obtain the target derived video data.

[0070] In yet another optional embodiment, the device may further include: The audio analysis module is used to obtain the original audio data recorded when the monitoring terminal captures the initial video data, extract the audio features in the original audio data and generate an audio feature set; determine whether there is a matching audio feature corresponding to a predetermined recognition target in the audio feature set; Furthermore, the target recognition module 201 determines whether there is a specific operation mode for recognizing a target in the initial video data according to the initial video data, which may include: The initial video data is identified based on the online monitoring recognition model. If the online monitoring recognition model identifies a recognition target from the initial video data, or determines that there is a matching audio feature in the audio feature set, it is determined that the recognition target exists in the initial video data.

[0071] Embodiment 3 See also Figure 3 , Figure 3 Schematic diagram of the structure of a video data acquisition system for a monitoring terminal disclosed in an embodiment of the present invention. Figure 3 As shown, the video data acquisition system of the monitoring terminal may include: A memory 301 storing executable program codes; a processor 302 coupled to the memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the video data acquisition method of the monitoring terminal described in the first embodiment of the present invention.

[0072] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0073] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0074] Finally, it should be noted that the video data acquisition method and device for a monitoring terminal disclosed in the embodiment of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for acquiring video data of a monitoring terminal, characterized in that: The method comprises: Acquire initial video data captured by the monitoring terminal, and determine whether there is an identification target in the initial video data according to the initial video data; When it is determined that the identification target does not exist in the initial video data, obtaining a target image in the initial video data, identifying the target image based on an online monitoring identification model, and obtaining an identification result of the online monitoring identification model; Determine whether the recognition result meets the preset model upgrade condition. If the recognition result meets the preset model upgrade condition, perform an upgrade operation on the online monitoring recognition model according to the target image to obtain an upgraded online monitoring recognition model; The upgraded online monitoring recognition model is used to perform intelligent recognition operations on the initial video data captured by the monitoring terminal to obtain target video data.

2. The video data acquisition method of the monitoring terminal according to claim 1, characterized in that: The step of identifying the target image based on the online monitoring recognition model and obtaining the recognition result of the online monitoring recognition model includes: Dividing the target image into multiple sub-areas according to a preset division strategy; Inserting a preset pattern to be identified in each sub-region, for each sub-region, determining a pixel region within a preset range around an edge position of the pattern to be identified as a transition region, performing a Gaussian filter operation on the transition region, and obtaining a target processing image; Recognize the target processing image based on the online monitoring recognition model, and obtain a pattern recognition result of the online monitoring recognition model, wherein the pattern recognition result includes a pattern recognized by the online monitoring recognition model from the target processing image; And, judging whether the recognition result meets the preset model upgrade condition includes: Comparing the pattern recognition result with the pattern to be recognized, and calculating the recognition accuracy of the pattern recognition result; If the accuracy of the pattern recognition result is lower than a preset accuracy threshold, it is determined that the recognition result meets a preset model upgrade condition.

3. The video data acquisition method of the monitoring terminal according to claim 2, characterized in that: The step of dividing the target image into a plurality of sub-areas according to a preset division strategy includes: Determining the distances from a plurality of reference points preset in the target image to the monitoring terminal according to the lens parameters corresponding to the monitoring terminal and the target image; Divide all reference points into a plurality of sub-reference point sets, wherein, in each of the sub-reference point sets, the distances from all the sub-reference points to the monitoring terminal are within a preset distance range corresponding to the sub-reference point set; According to all the sub-reference point sets, the target image is divided into a plurality of sub-regions, wherein each of the sub-reference point sets corresponds to a sub-region.

4. The video data acquisition method of the monitoring terminal according to claim 3 is characterized in that: The performing of a Gaussian filtering operation on the transition region to obtain a target processed image includes: Determining Gaussian filter algorithm parameters according to the preset distance range corresponding to each sub-area; A Gaussian filtering operation is performed on the transition area according to Gaussian filtering algorithm parameters corresponding to the transition area to obtain a target processed image.

5. The video data acquisition method of the monitoring terminal according to claim 1, characterized in that: The upgraded online monitoring recognition model is used to perform intelligent recognition operations on the initial video data captured by the monitoring terminal to obtain target video data, including: Using the upgraded online monitoring recognition model, an intelligent recognition operation is performed on the initial video data captured by the monitoring terminal to obtain an intelligent recognition result; Based on the intelligent recognition result, extracting a target character from a pre-loaded static character library, wherein the target character is used to represent the intelligent recognition result in a preset expression; The target character is superimposed on the video stream corresponding to the initial video data through a rendering operation to obtain target video data.

6. The video data acquisition method of the monitoring terminal according to claim 1 or 5, characterized in that: The method further comprises: Pulling a target video stream corresponding to the target video data from the monitoring terminal, decoding the target video stream according to a preset decoding scheme, and obtaining a decoded target video stream; According to the decoded target video stream, multiple derivative video streams corresponding to the target video stream are obtained according to a preset data multiplexing scheme, and according to a coding scheme corresponding to the preset decoding scheme, a re-encoding operation is performed on each derivative video stream to obtain multiple derivative video data.

7. The video data acquisition method of the monitoring terminal according to claim 6, characterized in that: The upgraded online monitoring recognition model is used to perform intelligent recognition operations on the initial video data captured by the monitoring terminal to obtain target video data, including: Using the upgraded online monitoring recognition model, an intelligent recognition operation is performed on the initial video data captured by the monitoring terminal to obtain an intelligent recognition result; Based on the intelligent recognition result, extracting a derived target character corresponding to each channel of derived video data from a pre-loaded static character library, wherein the derived target character corresponds to the resolution and bit rate of the corresponding derived video data, and the derived target character is used to represent the intelligent recognition result in a preset expression; For each channel of the derived video data, the derived target characters are superimposed on the video stream corresponding to the derived video data through a rendering operation to obtain target derived video data.

8. The video data acquisition method of the monitoring terminal according to claim 1, characterized in that: The method further comprises: Acquire original audio data recorded when the monitoring terminal captures initial video data, extract audio features from the original audio data and generate an audio feature set; Determining whether there is a matching audio feature corresponding to a predetermined recognition target in the audio feature set; And, judging whether there is an identification target in the initial video data according to the initial video data, comprising: The initial video data is identified based on an online monitoring recognition model. If the online monitoring recognition model identifies a recognition target from the initial video data, or determines that the matching audio feature exists in the audio feature set, it is determined that the recognition target exists in the initial video data.

9. A video data acquisition device for a monitoring terminal, characterized in that: The device includes: The target recognition module is used to obtain the initial video data captured by the monitoring terminal, and determine whether there is an identification target in the initial video data according to the initial video data; A model testing module, for obtaining a target image in the initial video data when it is determined that the identification target does not exist in the initial video data, identifying the target image based on the online monitoring identification model, and obtaining an identification result of the online monitoring identification model; A model upgrade module, used to determine whether the recognition result meets the preset model upgrade condition, and if the recognition result meets the preset model upgrade condition, perform an upgrade operation on the online monitoring recognition model according to the target image to obtain an upgraded online monitoring recognition model; The intelligent recognition module is used to perform intelligent recognition operations on the initial video data captured by the monitoring terminal through the upgraded online monitoring recognition model to obtain target video data.

10. A video data acquisition system for a monitoring terminal, characterized in that: The system includes: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the video data acquisition method of the monitoring terminal according to any one of claims 1-8.

Images