Method and device for determining prompt information in video surveillance

Abstract

The present application discloses a method and device for displaying information in video surveillance, which belongs to the field of video processing. The method comprises: acquiring a video screen, and selecting a predefined orientation whose orientation deviation from a target orientation is less than a preset orientation deviation from a plurality of predefined orientation images one by one. Since the target orientation is the acquisition orientation of the video screen, and the orientation deviation is less than the preset orientation deviation, it can be indicated that the target orientation and the predefined orientation are basically close. At this time, the prompt information set for the selected predefined orientation can be determined as the prompt information that needs to be superimposed in the video screen. The prompt information set for the selected predefined orientation does not need to be determined as the prompt information that needs to be superimposed in the video screen only when the state parameters of the target orientation and the predefined orientation are completely consistent, thereby improving the flexibility of determining prompt information in video surveillance.

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to a method and device for determining prompt information in video surveillance. Background Art

[0002] At present, video surveillance mainly uses a rotatable camera to collect video images in different directions to achieve monitoring of different directions. Since a rotatable camera is a camera that can dynamically change the viewing area, that is, the rotatable camera can be used to monitor any direction within the spatial range. Therefore, when a user uses a rotatable camera to monitor a target object of interest, the rotatable camera needs to be rotated to the direction where the target object is located. During the rotation of the rotatable camera, in order to facilitate the user to quickly find the direction where the target object is located, prompt information can be superimposed on the video images of certain directions of the rotatable camera. The prompt information is used to prompt the object in the video image or the current direction of the rotatable camera, such as the prompt information can be "West Railway Station" or "Current Direction South Direction".

[0003] In the related art, a plurality of preset positions are pre-set in a rotatable camera, and the state parameters and prompt information of each preset position are stored in the rotatable camera, and the state parameters include the spatial direction vector and zoom factor of the preset position. When the rotatable camera is rotating, the state parameters of the current position are detected in real time. When the detected state parameters are consistent with the state parameters of any preset position in the plurality of preset positions, the prompt information pre-set for the preset position is superimposed on the current captured video screen. The video screen after the superimposed prompt information is sent to a monitoring system for viewing the video screen, so that the monitoring system displays the video screen after the superimposed prompt information.

[0004] During the rotation of the above-mentioned rotatable camera, only when the state parameters of the current position are completely consistent with the state parameters of the preset position, the rotatable camera will superimpose the prompt information set for the preset position in the current video picture. That is, only when the spatial direction vector and zoom factor of the current position are completely consistent with the spatial direction vector and zoom factor of the preset position, the rotatable camera will determine the prompt information set for the preset position as the prompt information that needs to be superimposed in the current video picture. Therefore, when the target object exists in the video picture captured by the rotatable camera at the current position, but the state parameters of the current position are not completely consistent with the state parameters of the preset position, the rotatable camera will not superimpose the prompt information in the current video picture, resulting in the monitoring system not displaying the prompt information at this time. Summary of the invention

[0005] In order to solve the problem in the related art that when a target object exists in the video picture captured by a rotatable camera at the current position, but the state parameters of the current position are not completely consistent with the state parameters of the preset position, the rotatable camera will not superimpose prompt information in the current video picture, resulting in the monitoring system not displaying the prompt information at this time, the present application provides a method and device for determining prompt information in video monitoring. The technical solution is as follows:

[0006] In a first aspect, a method for determining prompt information in video surveillance is provided, the method comprising:

[0007] In the process of finding the location of the target object, a video image is obtained;

[0008] Based on the video picture and a plurality of predefined directions stored in advance, a predefined direction close to a target direction is selected from the plurality of predefined directions, wherein the target direction is the acquisition direction of the video picture;

[0009] The prompt information set for the selected predefined orientation is determined as the prompt information of the video picture of the target orientation.

[0010] In a second aspect, a device for determining prompt information in video surveillance is provided, the device comprising:

[0011] In the process of finding the location of the target object, the acquisition module is used to acquire the video picture;

[0012] A first determining module, configured to determine a plurality of pre-stored predefined locations;

[0013] A selection module, configured to select a predefined orientation close to a target orientation from the plurality of predefined orientations based on the video image, wherein the target orientation is a collection orientation of the video image;

[0014] The second determining module is used to determine the prompt information set for the selected predefined orientation as the prompt information of the video picture of the target orientation.

[0015] In a third aspect, a device for displaying information in video surveillance is provided, wherein the structure of the device for displaying information in video surveillance includes a processor and a memory, wherein:

[0016] The memory is used to store programs;

[0017] The processor is configured to execute the program stored in the memory to perform:

[0018] In the process of finding the location of the target object, a video image is obtained;

[0019] Based on the video picture and a plurality of pre-stored predefined directions, a predefined direction close to a target direction is selected from the plurality of predefined directions, wherein the target direction is a collection direction of the video picture;

[0020] The prompt information set for the selected predefined orientation is determined as the prompt information of the video picture of the target orientation.

[0021] In a fourth aspect, a method for determining prompt information in video surveillance is provided, the method comprising:

[0022] Get the video screen;

[0023] Based on the video picture and a plurality of predefined orientation images stored in advance, determining an orientation deviation between a target orientation and each of the plurality of predefined orientations, wherein the target orientation is a collection orientation of the video picture, and the plurality of predefined orientation images correspond one-to-one to the plurality of predefined orientations;

[0024] Selecting a predefined direction from the plurality of predefined directions, the direction deviation of which from the target direction is less than a preset direction deviation;

[0025] The prompt information set for the selected predefined orientation is determined as the prompt information that needs to be superimposed in the video picture.

[0026] In an embodiment of the present invention, a video picture is acquired, and a predefined direction whose direction deviation from a target direction is less than a preset direction deviation is selected from a plurality of predefined directions corresponding one to one with a plurality of predefined direction images. Since the target direction is the acquisition direction of the video picture, and the direction deviation is less than the preset direction deviation, it can be indicated that the target direction and the predefined direction are basically close. At this time, the rotatable camera can determine the prompt information set for the selected predefined direction as the prompt information that needs to be superimposed in the video picture, without the need to determine the prompt information set for the selected predefined direction as the prompt information that needs to be superimposed in the video picture only when the state parameters of the target direction and the predefined direction are completely consistent.

[0027] Optionally, determining the orientation deviation between the target orientation and each of the plurality of predefined orientations based on the video image and the plurality of predefined orientation images stored in advance comprises:

[0028] For each of the plurality of predefined orientation images, obtaining a plurality of pairs of local feature points that match the predefined orientation image and the video picture, each pair of local feature points being edge points or corner points that match the video picture and the predefined orientation image;

[0029] When the number of the multiple pairs of local feature points is greater than a preset number, an orientation deviation between a predefined orientation corresponding to the predefined orientation image and the target orientation is determined according to the multiple pairs of local feature points.

[0030] Specifically, the rotatable camera may determine the orientation deviation between the predefined orientation and the target orientation according to the predefined orientation image and the matching local feature points in the video picture.

[0031] Optionally, the predefined orientation image is an image including first-order edge features;

[0032] The obtaining of a plurality of pairs of local feature points matching the predefined orientation image and the video picture includes:

[0033] Determine the background image of the video screen by using a Gaussian mixture background modeling method;

[0034] Processing the background image of the video screen to obtain a processed image, wherein the processed image is an image including first-order edge features;

[0035] The feature points in the processed image and the feature points in the predefined orientation image are matched by a scale-invariant feature transform (SIFT) algorithm to obtain a plurality of matched pairs of local feature points.

[0036] Furthermore, in order to facilitate the search for matching local feature points in the predefined orientation image and the video screen, the video screen may be processed to obtain a processed image, and then multiple pairs of matching local feature points may be determined from the processed image and the predefined orientation image.

[0037] Optionally, determining the orientation deviation between a predefined orientation corresponding to the predefined orientation image and the target orientation according to the multiple pairs of local feature points includes:

[0038] Selecting at least three pairs of local feature points from the multiple pairs of local feature points;

[0039] At least one pair of triangles is constructed by any three pairs of local feature points among the at least three pairs of local feature points, each pair of triangles includes a first triangle and a second triangle, the first triangle is a triangle constructed by the local feature points in the video picture, the second triangle is a triangle constructed by the local feature points in the predefined orientation image, and each side length of the first triangle and the second triangle is greater than a preset side length, and each internal angle of the first triangle and the second triangle is greater than a preset angle;

[0040] For each pair of triangles in the at least one pair of triangles, determining a sum of squares of differences between interior angles corresponding to matching local feature points in a first triangle and a second triangle included in the pair of triangles;

[0041] When the square sum is greater than a preset value, spatially transforming the first triangle so that the square sum of the differences between the inner angles corresponding to the local feature points matched in the spatially transformed triangle and the second triangle is less than the preset value, and the size of the spatially transformed triangle and the second triangle is the same;

[0042] Determine a ratio between the field of view deflection angle of the triangle after the spatial transformation and the field of view deflection angle of the second triangle, where the field of view deflection angle is the angle formed by the lines connecting the center of gravity of the triangle and the center of the image where the triangle is located and the focus of the rotatable camera respectively;

[0043] According to the determined at least one ratio, an orientation deviation between a predefined orientation corresponding to the predefined orientation image and the target orientation is determined.

[0044] Specifically, at least one pair of triangles may be constructed according to multiple pairs of local feature points, and then the orientation deviation between the predefined orientation and the target orientation may be determined according to the at least one pair of constructed triangles.

[0045] Optionally, performing spatial transformation on the first triangle includes:

[0046] Performing an affine transformation on the first triangle so that the sum of squares of differences between the inner angles corresponding to the local feature points matched in the affine-transformed triangle and the second triangle is less than the preset value;

[0047] Determine an average value of the ratios of the corresponding side lengths of the triangle after the affine transformation and the second triangle;

[0048] When the average value is greater than 1, the triangle after the affine transformation is reduced by taking the average value as the reduction ratio to obtain the triangle after the spatial transformation;

[0049] When the average value is not greater than 1, the triangle after the affine transformation is enlarged with the reciprocal of the average value as the enlargement ratio to obtain the triangle after the spatial transformation.

[0050] The first triangle is spatially transformed so that the first triangle after spatial transformation is as similar as possible to the second triangle.

[0051] Optionally, determining the ratio between the field of view deflection angle of the triangle after the spatial transformation and the field of view deflection angle of the second triangle includes:

[0052] Determine the distance between the center of the spatially transformed triangle and the center of the video picture where the spatially transformed triangle is located to obtain a first distance;

[0053] determining a distance between a center of the second triangle and a center of an image where the second triangle is located, to obtain a second distance;

[0054] The ratio between the first distance and the second distance is determined as the ratio between the field of view deflection angle of the triangle after the spatial transformation and the field of view deflection angle of the second triangle.

[0055] Since the viewing angle is not easy to measure directly, the above indirect method can be used to determine the ratio between the viewing angle of the triangle after spatial transformation and the viewing angle of the second triangle.

[0056] Optionally, determining the orientation deviation between the predefined orientation corresponding to the predefined orientation image and the target orientation according to the determined at least one ratio includes:

[0057] Performing linear clustering calculation on the at least one ratio to obtain a discrete value of each ratio;

[0058] Selecting a ratio whose discrete value is smaller than a preset discrete value from the at least one ratio;

[0059] An absolute value of a difference between an average value of the selected ratios and 1 is determined as an orientation deviation between a predefined orientation corresponding to the predefined orientation image and the target orientation.

[0060] In order to avoid the existence of relatively discrete ratios in the at least one ratio, the relatively discrete ratios in the at least one ratio may be discarded.

[0061] Optionally, before determining the orientation deviation between the target orientation and each of the plurality of predefined orientations based on the video image and the plurality of pre-stored predefined orientation images, the method further includes:

[0062] When a predefined orientation setting instruction is detected, the current orientation is set to the predefined orientation, and the video picture of the current orientation is set to the video picture of the predefined orientation;

[0063] Determine the background image of the video picture at the predefined position by using a Gaussian mixture background modeling method;

[0064] Processing the background image of the video screen at the predefined orientation to obtain a processed image, wherein the processed image is an image including a first-order edge feature;

[0065] The processed image is determined as a predefined orientation image corresponding to the predefined orientation, and the predefined orientation image is stored.

[0066] In the embodiment of the present invention, a predefined orientation image may be preset and the preset predefined orientation image may be stored.

[0067] In a fifth aspect, a device for displaying information in video surveillance is provided, wherein the device for displaying information in video surveillance has the function of implementing the method for displaying information in video surveillance in the first aspect. The device for displaying information in video surveillance includes at least one module, and the at least one module is used to implement the method for displaying information in video surveillance provided in the first aspect.

[0068] In a sixth aspect, a device for displaying information in video surveillance is provided, wherein the structure of the device for displaying information in video surveillance includes a processor and a memory, wherein the memory is used to store a program that supports the device for displaying information in video surveillance to execute the method for displaying information in video surveillance provided in the first aspect, and to store data involved in implementing the method for displaying information in video surveillance provided in the first aspect. The processor is configured to execute the program stored in the memory. The operating device of the storage device may also include a communication bus, which is used to establish a connection between the processor and the memory.

[0069] In a seventh aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the information display method in video surveillance described in the first aspect above.

[0070] In an eighth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the information display method in video surveillance described in the first aspect.

[0071] The technical effects obtained by the above-mentioned second, third, fourth and fifth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here.

[0072] The technical solution provided by the present application has the following beneficial effects: obtaining a video screen, and selecting a predefined orientation whose orientation deviation from the target orientation is less than the preset orientation deviation from a plurality of predefined orientation images. Since the target orientation is the acquisition orientation of the video screen, and the orientation deviation is less than the preset orientation deviation, it can be shown that the target orientation and the predefined orientation are basically close. At this time, the prompt information set for the selected predefined orientation can be determined as the prompt information that needs to be superimposed in the video screen, so that the monitoring system displays the prompt information. The prompt information set for the selected predefined orientation does not need to be determined as the prompt information that needs to be superimposed in the video screen only when the state parameters of the target orientation and the predefined orientation are completely consistent, which improves the flexibility of determining prompt information in video monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a schematic diagram of a video monitoring system provided by an embodiment of the present invention;

[0074] Figure 2 is a structural schematic diagram of a rotatable camera provided by an embodiment of the present invention;

[0075] Figure 3A It is a flow chart of a method for determining prompt information in video surveillance provided by an embodiment of the present invention;

[0076] Figure 3B is a distribution diagram of a pixel point and pixel points around the pixel point provided by an embodiment of the present invention;

[0077] Figure 4A is a flow chart of another method for determining prompt information in video surveillance provided by an embodiment of the present invention;

[0078] Figure 4B is a schematic diagram of a first triangle and a second triangle provided by an embodiment of the present invention;

[0079] Figure 4C is a schematic diagram of a triangle and a second triangle after affine transformation provided by an embodiment of the present invention;

[0080] Figure 4D A schematic diagram of a field of view deflection angle provided by an embodiment of the present invention;

[0081] Figure 5 is a flow chart of another method for determining prompt information in video surveillance provided by an embodiment of the present invention;

[0082] Fig. 6A It is a block diagram of a device for determining prompt information in video monitoring provided by an embodiment of the present invention;

[0083] Figure 6BThis is a block diagram of another device for determining prompt information in video surveillance provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0085] Before explaining the embodiment of the present invention in detail, the application scenario of the embodiment of the present invention is first introduced. In the related art, in order to facilitate the user to quickly find the location of the target object through a rotatable camera, when the rotatable camera is rotating, prompt information can be displayed in the video picture collected during the rotation of the rotatable camera, so that the user can understand the approximate location of the camera of the current rotatable camera. For example, the prompt information "The current direction is due south" is displayed in the video picture collected by the rotatable camera at the current direction. When the user views the prompt information, he knows that the current direction of the rotatable camera is due south. The method for determining prompt information in video surveillance provided by the embodiment of the present invention is applied to the scenario where the rotatable camera displays prompt information during the rotation process.

[0086] Figure 1 Schematic diagram of a video monitoring system provided by an embodiment of the present invention. Figure 1 As shown, the video monitoring system includes a rotatable camera 101 and a monitoring system 102, and the monitoring system 102 includes a monitoring server 1021 and a monitoring client 1022. The rotatable camera 101 and the monitoring server 1021 can communicate with each other via a wireless network or a wired network. The rotatable camera 101 and the monitoring client 1022 can communicate with each other via a wireless network or a wired network. The monitoring server 1021 and the monitoring client 1022 can also communicate with each other via a wireless network or a wired network.

[0087] The rotatable camera 101 is used to collect video images from different directions and send the collected video images to the monitoring server 1021 and / or the monitoring client 1022. The monitoring client 1022 is used to receive the video images sent by the rotatable camera 101 or the monitoring server 1021, and display the video images so that the user can view the video images collected by the rotatable camera 101 through the monitoring client 1022. The monitoring server 1021 is used to receive the video images collected by the rotatable camera 101, process the video images, and send the processed video images to the monitoring client 1022.

[0088] In actual application, the monitoring client 1022 may be an administrator client. At this time, the administrator may control the rotatable camera 101 to perform certain operations such as rotation through the monitoring client 1022, or set certain information for the rotatable camera through the monitoring client, such as setting a predefined orientation or setting prompt information.

[0089] Figure 2 is a schematic diagram of a rotatable camera structure provided by an embodiment of the present invention. The rotatable camera can be Figure 1 The rotatable camera 101 is shown. In practical applications, the rotatable camera can be a pan-tilt-zoom (PTZ) camera. Figure 2 The rotatable camera includes: a video collector 201, a transmitter 202, a receiver 203, a memory 204, and a processor 205.

[0090] The video collector 201 is used to collect video images, the transmitter 202 is used to send data and / or signaling, etc. The receiver 203 is used to receive data and / or signaling, etc.

[0091] The memory 204 may be used to store one or more software programs and / or modules. The memory may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by the integrated circuit, but not limited thereto.

[0092] The processor 205 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0093] In a specific implementation, as an embodiment, the rotatable camera may further include an output device and an input device. The output device communicates with the processor 205 and may display information in a variety of ways. For example, the output device may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device communicates with the processor 205 and may receive user input in a variety of ways. For example, the input device may be a mouse, a keyboard, a touch screen device, or a sensor device.

[0094] The memory 204 is used to store program codes for executing the solution of the present application, and the execution is controlled by the processor 205. The processor 205 is used to execute the program codes stored in the memory 204. The program codes may include one or more software modules. Figure 1 The video surveillance system shown in FIG. 2 can determine data for developing an application through the processor 205 and one or more software modules in the program code in the memory 204 .

[0095] It should be noted that the method for determining prompt information in video surveillance provided by the embodiment of the present invention mainly includes two aspects: one is to set multiple predefined orientation images for the rotatable camera, and the other is to determine whether to determine the prompt information set for the predefined orientation as the prompt information that needs to be superimposed in the video image based on the video image and the multiple predefined orientation images. The following embodiments of the present invention will respectively discuss these two aspects.

[0096] First, the process of setting multiple predefined orientation images for a rotatable camera is described in detail. It is worth noting that the process of setting multiple predefined orientation images can be Figure 1 This can be performed by a rotatable camera in Figure 1 The following embodiment takes the process of presetting a plurality of predefined orientation images for a rotatable camera as an example for explanation.

[0097] Figure 3A A method for determining prompt information in video surveillance provided by an embodiment of the present invention is applied to Figure 1 In the video surveillance system shown in the figure, the embodiment of the present invention will describe in detail the process of presetting multiple predefined orientation images for a rotatable camera. Figure 3A , the method comprises the following steps.

[0098] Step 301: When the rotatable camera detects a predefined orientation setting instruction, the current orientation is set to the predefined orientation, and the video picture at the current orientation is set to the video picture at the predefined orientation.

[0099] When the administrator wants to set multiple predefined orientation images for the rotatable camera through the monitoring client, the administrator can control the rotatable camera to rotate through the monitoring client. That is, the monitoring client sends a rotation request to the rotatable camera, and when the rotatable camera receives the rotation request, it rotates, and collects video images in real time during the rotation process, and sends the real-time collected video images to the monitoring client, and the monitoring client displays the real-time received video images. When the monitoring client detects a predefined orientation setting instruction during the process of displaying the video image, it forwards the predefined orientation setting instruction to the rotatable camera.

[0100] The predefined location instruction is triggered by an administrator through a preset operation, and the preset operation may be a click operation, a slide operation, or a voice operation.

[0101] Step 302: The rotatable camera determines the background image of the video screen at the predefined orientation by using a Gaussian mixture background modeling method.

[0102] Since the video picture at the predefined orientation captured by the rotatable camera includes a foreground image and a background image, and the foreground image is usually an image of a non-stationary object, such as an image of a moving vehicle, in order to make the predefined orientation image and the video picture captured by the rotatable camera at the predefined orientation at different times as consistent as possible, the rotatable camera needs to obtain the background image of the video picture at the predefined orientation.

[0103] For the convenience of subsequent explanation, the Gaussian mixture background modeling method is introduced here.

[0104] Since the brightness difference between the foreground image and the background image in the same picture is large, and the brightness of the foreground image and the brightness of the background image both obey Gaussian distribution, there will be a double peak-valley shape in the brightness distribution histogram of the same image, in which one peak corresponds to the brightness distribution of the foreground image, and the other peak corresponds to the brightness distribution of the background image. When the rotatable camera collects video images of a predefined orientation, the collected video images of the predefined orientation are multi-frame video images, so the rotatable camera can obtain the brightness distribution of multiple background images based on the multi-frame video images, and establish a background image brightness distribution model, i.e., a Gaussian mixture model, based on the brightness distribution of the multiple background images. For any frame of the multi-frame video images, each pixel in the video image is matched with the above-mentioned Gaussian mixture model. When the brightness of the pixel matches the above-mentioned Gaussian mixture model, the pixel is determined to be a pixel in the background image. When the brightness of the pixel does not match the above-mentioned Gaussian mixture model, the pixel is determined to be a pixel in the foreground image. When the rotatable camera performs the above operation on all the pixels in the video picture, the pixels in the background image are obtained, that is, the background image of the video picture in the predefined orientation is determined.

[0105] It should be noted that the embodiment of the present invention may also determine the background image of the video screen of the predefined orientation by other methods such as a single Gaussian model method or an initialization method based on an image sequence, which is not specifically limited in the embodiment of the present invention.

[0106] Step 303: The rotatable camera processes the background image of the video screen in the predefined orientation to obtain a processed image, where the processed image is an image including first-order edge features.

[0107] Specifically, the background image of the video screen of the predefined orientation is grayed, that is, only the brightness value of each pixel is used to represent the characteristics of the pixel, to obtain a grayscale image of the background image, and Gaussian blur is performed on the grayscale image of the background image to remove local small features in the grayscale image, and Sobel transform is performed on the grayscale image after Gaussian blur to obtain an image including only first-order edge features.

[0108] Among them, Gaussian blur is performed on the grayscale image of the background image, that is, for each pixel in the grayscale image, the grayscale values ​​of a first preset number of pixels around the pixel are counted to obtain the average value of the grayscale value at the pixel in the grayscale image, and the grayscale value of the pixel is replaced by the average value of the grayscale value.

[0109] In addition, the grayscale image after Gaussian blurring is subjected to Sobel transformation, that is, for each pixel in the grayscale image after Gaussian blurring, a second preset number of pixels around the pixel are obtained, different weights are assigned to the second preset number of pixels, and the grayscale gradient of the pixel is determined according to the grayscale value of the second preset number of pixels. When the grayscale gradient of the pixel is greater than the gradient threshold, it indicates that there is a drastic change in the grayscale value at the pixel. From all the pixels of the grayscale image after Gaussian blurring, pixels whose grayscale gradient is greater than the gradient threshold are selected, and the image formed by the selected pixels is determined as the processed image. Since the processed image only includes pixels whose grayscale gradient is greater than the gradient threshold, the processed image can approximately characterize the contour of the background image of the video screen of the predefined orientation, that is, the processed image is an image including first-order edge features.

[0110] The grayscale gradient of a pixel is used to indicate the transformation between the grayscale value of the pixel and the grayscale values ​​of a second preset number of surrounding pixels. In a possible implementation, Figure 3B As shown, point P is the pixel point, points P1-P8 are the 8 pixel points around point P, and the grayscale gradient of point P in the vertical direction can be expressed as: (P6`+2P7`+P8`)-(P1`+2P2`+P3`), where P1`, P2`, P3`, P6`, P7` and P8` are the grayscale values ​​of pixel points P1, P2, P3, P6, P7 and P8 respectively.

[0111] Step 304: The rotatable camera determines the processed image as a predefined orientation image corresponding to the predefined orientation, and stores the predefined orientation image.

[0112] After storing the predefined orientation image, the rotatable camera can also establish a corresponding relationship between the predefined orientation and the predefined orientation image.

[0113] In addition, when the rotatable camera receives a predefined orientation setting instruction sent by the monitoring client, it also receives a prompt message for the predefined orientation setting sent by the monitoring client. At this time, the rotatable camera also needs to store the prompt message in the correspondence between the above-mentioned predefined orientation and the predefined orientation image, thereby obtaining a correspondence between the predefined orientation, the predefined orientation image and the prompt message.

[0114] For example, the administrator sets multiple predefined positions through the monitoring client, and marks them as predefined position 1, predefined position 2, predefined position 3, etc. The rotatable camera obtains the predefined position images of each of the multiple predefined positions, which are image A, image B, image C, etc., and receives the prompt information set for the multiple predefined positions sent by the monitoring client, which are information I, information II, information III, etc. At this time, the rotatable camera establishes the corresponding relationship shown in Table 1, wherein each predefined position corresponds to a predefined position image and a prompt information.

[0115] Table 1

[0116] Predefined locations Predefined position images Tips Predefined position 1 Image A Information I Predefined location 2 Image B Information II Predefined location 3 Image C Information III … … …

[0117] Optionally, when the monitoring server performs the process of setting multiple predefined orientation images, at this time, when the monitoring client receives the video images captured by the rotatable camera in real time, if it detects the predefined orientation setting instruction triggered by the administrator through the preset operation, the current orientation is determined as the predefined orientation, and the predefined orientation setting instruction is forwarded to the monitoring server, and the predefined orientation setting instruction carries the video image of the current orientation. When the monitoring server receives the predefined orientation setting instruction, it determines the predefined orientation image according to the above steps 301 to 303, obtains multiple predefined orientation images and prompt information set for each of the multiple predefined orientations, and establishes the corresponding relationship shown in Table 1, and sends the multiple predefined orientation images, the prompt information set for the multiple predefined orientations, and the corresponding relationship shown in Table 1 to the rotatable camera and the monitoring client, so that the rotatable camera and the monitoring client store the multiple predefined orientation images, the prompt information set for the multiple predefined orientations, and the corresponding relationship shown in Table 1.

[0118] In an embodiment of the present invention, multiple predefined orientations are set in advance for the rotatable camera, and predefined orientation images and prompt information corresponding to the multiple predefined orientations are determined one by one, so that according to the multiple predefined orientation images, it is determined whether the prompt information set for the predefined orientation is determined as the prompt information that needs to be superimposed in the video screen.

[0119] After setting a plurality of predefined orientations and predefined orientation images and prompt information corresponding to the plurality of predefined orientations for the rotatable camera, it is necessary to determine whether to determine the prompt information set for the predefined orientation as prompt information that needs to be superimposed in the video image based on the video image and the plurality of predefined orientation images. The following embodiment is used to explain the process in detail.

[0120] It should be noted that, according to the video screen and the plurality of predefined position images, the process of determining whether to determine the prompt information set for the predefined position as the prompt information that needs to be superimposed in the video screen can be performed by Figure 1 This can be performed by a rotatable camera in Figure 1 The following two embodiments will be used to explain these two situations in detail.

[0121] Figure 4A Another method for determining prompt information in video surveillance provided by an embodiment of the present invention is applied to Figure 1 In the video surveillance system shown, the embodiment of the present invention will describe in detail the process of determining whether to determine the prompt information set for the predefined orientation as the prompt information that needs to be superimposed in the video image based on the video image captured by the rotatable camera according to the target orientation and multiple predefined orientation images. Figure 4A , the method comprises the following steps.

[0122] Step 401: The camera can be rotated to obtain video images.

[0123] Specifically, when the user wants to control the rotatable camera to rotate to the position where the target object is located through the monitoring client, the monitoring client sends a rotation request to the rotatable camera. When the rotatable camera receives the rotation request, it rotates and collects video images in real time during the rotation process. At this time, the rotatable camera can determine the video image collected in real time as the video image.

[0124] Of course, in order to reduce the data processing pressure of the rotatable camera, the rotatable camera can also select a video picture from the video pictures collected in the preset time period closest to the current time before the current time every preset time period, and determine the selected video picture as the video picture.

[0125] For the convenience of subsequent explanation, the orientation at which the rotatable camera captures the video image is referred to as the target orientation, that is, the target orientation is the acquisition orientation of the video image.

[0126] In order to determine whether to superimpose the preset prompt information on the video screen, the rotatable camera needs to determine whether the target direction is consistent with Figure 3AOne of the multiple predefined orientations in the illustrated embodiment is close. Specifically, the rotatable camera can determine the orientation deviation between the target orientation and each of the multiple predefined orientations based on the video screen and the multiple predefined orientation images stored in advance, wherein the multiple predefined orientation images correspond to the multiple predefined orientations one by one. The following steps 402 to 408 will introduce in detail how the rotatable camera determines the orientation deviation between the target orientation and each of the multiple predefined orientations based on the video screen and the multiple predefined orientation images stored in advance.

[0127] Step 402: For each predefined orientation image in the multiple predefined orientation images, the camera can be rotated to obtain multiple pairs of local feature points that match the predefined orientation image and the video image, each pair of local feature points is an edge point or corner point that matches the video image and the predefined orientation image.

[0128] Depend on Figure 3A It can be seen from the illustrated embodiment that the predefined orientation image is an image including first-order edge features. Therefore, the implementation process of obtaining the predefined orientation image and multiple pairs of local feature points that match the video screen can be specifically as follows: determining the background image of the video screen by a Gaussian mixture background modeling method; processing the background image of the video screen to obtain a processed image, which is an image including first-order edge features; matching the feature points in the processed image with the feature points in the predefined orientation image by a scale invariant feature transform (SIFT) algorithm to obtain multiple pairs of matching local feature points.

[0129] The video image is processed to obtain the processed image. Figure 3A The implementation process of step 302 and step 303 is basically the same, and the embodiment of the present invention will not be elaborated in detail here.

[0130] In addition, the feature points in the processed image and the feature points in the predefined orientation image are matched by the SIFT algorithm to obtain multiple pairs of matched local feature points. Specifically, for each feature point in the processed image and the predefined orientation image, according to Figure 3AThe method for determining the longitudinal grayscale gradient of the pixel point in step 303 determines the transverse grayscale gradient and the longitudinal grayscale gradient of the feature point, and uses the vector formed by the transverse grayscale gradient and the longitudinal grayscale gradient of the feature point to represent the direction angle of the feature point. For any feature point in the processed image, the Euclidean distance between the direction angle of the feature point and the azimuth angle of each feature point included in the predefined orientation image is determined to obtain multiple Euclidean distances. If a Euclidean distance less than the Euclidean distance threshold is found from the multiple Euclidean distances, the feature point corresponding to the found Euclidean distance is determined as a feature point matching any feature point in the processed image. The Euclidean distance is the real distance between two points in the multidimensional space.

[0131] After obtaining the matched multiple pairs of local feature points, the rotatable camera determines the number of the multiple pairs of local feature points. When the number of the multiple pairs of local feature points is greater than the preset number, it indicates that the processed image and the predefined orientation image are relatively similar. At this time, the rotatable camera can determine the orientation deviation between the predefined orientation corresponding to the predefined orientation image and the target orientation according to the multiple pairs of local feature points. Specifically, the rotatable camera can determine the orientation deviation between the predefined orientation and the target orientation through steps 403 to 408.

[0132] Step 403: The rotatable camera selects at least three pairs of local feature points from the multiple pairs of local feature points.

[0133] Each pair of local feature points in the at least three selected pairs of local feature points should satisfy the conditions in step 404. The specific conditions are detailed in step 404 and will not be elaborated in detail here.

[0134] Step 404: The rotatable camera constructs at least one pair of triangles through any three pairs of local feature points among the at least three pairs of local feature points, each pair of triangles includes a first triangle and a second triangle, the first triangle is a triangle constructed by the local feature points in the video picture, and the second triangle is a triangle constructed by the local feature points in the predefined orientation image.

[0135] In order to facilitate the rotatable camera to process the first triangle and the second triangle, the first triangle and the second triangle should meet the following conditions: each side length of the first triangle and the second triangle is greater than a preset side length, and each internal angle of the first triangle and the second triangle is greater than a preset angle.

[0136] In addition, since the length in an image is usually represented by a pixel value, the preset side length may be a preset pixel value.

[0137] Step 405: For each pair of triangles in the at least one pair of triangles, the rotatable camera determines the sum of squares of differences between the inner angles corresponding to the matched local feature points in the first triangle and the second triangle included in the pair of triangles.

[0138] Figure 4B A first triangle and a second triangle are provided for an embodiment of the present invention, wherein the first triangle is △ABC and the second triangle is △A`B`C`. The vertex A in the first triangle and the vertex A` in the second triangle are a pair of matching local feature points, the vertex B in the first triangle and the vertex B` in the second triangle are a pair of matching local feature points, the vertex C in the first triangle and the vertex C` in the second triangle are a pair of matching local feature points, the internal angles corresponding to the vertices A, B and C in △ABC are α1, α2 and α3 respectively, and the internal angles corresponding to the vertices A`, B` and C` in △A`B`C` are β1, β2 and β3 respectively.

[0139] like Figure 4B As shown, the sum of the squares of the differences between the inner angles corresponding to the matched local feature points in the first triangle and the second triangle can be expressed by the following formula:

[0140] (α1-β1) 2 +(α2-β2) 2 +(α3-β3) 2

[0141] Among them, the sum of squares can represent the degree of deviation between the orientation of the video picture where the first triangle is located and the orientation of the video picture where the second triangle is located, that is, when the sum of squares is greater than a preset value, it indicates that the deviation between the orientation of the video picture where the first triangle is located and the orientation of the video picture where the second triangle is located, which is larger, and the rotatable camera executes step 406 at this time; when the sum of squares is not greater than the preset value, it indicates that the deviation between the orientation of the video picture where the first triangle is located and the orientation of the video picture where the second triangle is located, which is smaller, and the rotatable camera can directly execute step 407 at this time.

[0142] Step 406: When the sum of the squares is greater than a preset value, the camera can be rotated to spatially transform the first triangle so that the sum of the squares of the differences between the inner angles corresponding to the matching local feature points in the spatially transformed triangle and the second triangle is less than the preset value, and the size of the spatially transformed triangle and the second triangle is the same.

[0143] Specifically, the first triangle is affine transformed so that the sum of the squares of the differences between the inner angles corresponding to the matching local feature points in the affine transformed triangle and the second triangle is less than the preset value; the average value of the ratios of the corresponding side lengths in the affine transformed triangle and the second triangle is determined; when the average value is greater than 1, the affine transformed triangle is reduced by using the average value as a reduction ratio to obtain the spatially transformed triangle; when the average value is not greater than 1, the affine transformed triangle is enlarged by using the inverse of the average value as an enlargement ratio to obtain the spatially transformed triangle.

[0144] The process of affine transforming the first triangle may be as follows: determining an affine transformation matrix according to the three pairs of local feature points matched in the first triangle and the second triangle. Affine transforming the first triangle according to the affine transformation matrix to obtain a triangle after the first affine transformation. And judging whether the sum of the squares of the differences between the inner angles corresponding to the local feature points matched in the triangle after the first affine transformation and the second triangle is less than the preset value, when the sum of the squares of the differences between the inner angles corresponding to the local feature points matched in the triangle after the first affine transformation and the second triangle is not less than the preset value, affine transforming the triangle after the first affine transformation again according to the affine transformation matrix to obtain a triangle after the second affine transformation, and repeating the above operation until the sum of the squares of the differences between the inner angles corresponding to the local feature points matched in the triangle after the nth affine transformation and the second triangle is less than the preset value, and determining the triangle after the nth affine transformation as the triangle after the affine transformation. According to the three pairs of local feature points matched in the first triangle and the second triangle, the determination of the affine transformation matrix can refer to the affine transformation related technology, which will not be elaborated in detail here.

[0145] Figure 4C The affine transformed triangle and the second triangle provided by an embodiment of the present invention are as follows: Figure 4C As shown, the triangle after the affine transformation is △A``B``C``, and the second triangle is △A`B`C`. Among them, the vertex A`` in the triangle after the affine transformation and the vertex A` in the second triangle are a pair of matching local feature points, the vertex B`` in the triangle after the affine transformation and the vertex B` in the second triangle are a pair of matching local feature points, the vertex C`` in the triangle after the affine transformation and the vertex C` in the second triangle are a pair of matching local feature points, the three side lengths of △A``B``C`` are L1, L2 and L3 respectively, and the three side lengths corresponding to the side lengths L1, L2 and L3 in △A`B`C` are R1, R2 and R3 respectively.

[0146] At this time, the average value of the ratio of the corresponding side lengths of the triangle after affine transformation and the second triangle can be expressed by the following formula:

[0147] Ratio=AVG(L1 / R1, L2 / R2, L3 / R3),

[0148] Among them, AVG represents taking the average value, and Ratio represents the average value of the ratio of the corresponding side lengths of the triangle after affine transformation and the second triangle.

[0149] For example, when the Ratio is 2, the affine transformed triangle is reduced by two times to obtain the spatially transformed triangle; when the Ratio is 0.5, the affine transformed triangle is enlarged by two times to obtain the spatially transformed triangle, so that the spatially transformed triangle and the second triangle have the same size.

[0150] Step 407: The rotatable camera determines the ratio between the field of view deflection angle of the triangle after the spatial transformation and the field of view deflection angle of the second triangle, where the field of view deflection angle is the angle formed by the lines connecting the center of gravity of the triangle and the center of the image where the triangle is located, and the focus of the rotatable camera.

[0151] Since the field of view deviation angle of a rotatable camera when shooting a target object can represent the orientation of the rotatable camera when shooting the target object to a certain extent, the degree of deviation between the target orientation and the predefined orientation can be determined by determining the ratio between the field of view deviation angle of the triangle after the spatial transformation and the field of view deviation angle of the second triangle.

[0152] In addition, since it is difficult to directly measure the size of the field of view deviation angle, the ratio between the field of view deviation angle of the spatially transformed triangle and the field of view deviation angle of the second triangle can be determined by an indirect method. Specifically, it can be: determine the distance between the center of the spatially transformed triangle and the center of the video screen where the spatially transformed triangle is located, to obtain a first distance; determine the distance between the center of the second triangle and the center of the image where the second triangle is located, to obtain a second distance; determine the ratio between the first distance and the second distance as the ratio between the field of view deviation angle of the spatially transformed triangle and the field of view deviation angle of the second triangle.

[0153] Figure 4D A schematic diagram of a field of view deflection angle provided by an embodiment of the present invention is shown in FIG. Figure 4DAs shown in Figure (a), triangle △A```B```C``` is the triangle after the spatial transformation, the center of △A```B```C``` is point G, plane α is the video screen where the triangle after the spatial transformation is located, the center of plane α is point O, and point J is the focus of the rotatable camera. Therefore, the angle θ in Figure (a) is the field of view deflection angle of the triangle after the spatial transformation, and the first distance is M1. Figure 4D As shown in Figure (b), triangle △A`B`C` is the second triangle, the center of △A`B`C` is point G`, plane β is the image of the second triangle, the center of plane β is point O`, and point J is the focus of the rotatable camera. Therefore, the angle θ` in Figure (b) is the field of view deflection angle of the second triangle, and the second distance is M2. Figure 4D As shown, the ratio between the field of view deviation angle of the triangle after spatial transformation and the field of view deviation angle of the second triangle is θ / θ`. Since θ / θ`=M1 / M2, the ratio of the first distance to the second distance, i.e., M1 / M2, can be determined as the ratio between the field of view deviation angle of the triangle after spatial transformation and the field of view deviation angle of the second triangle.

[0154] When the rotatable camera performs the operations of step 405, step 406 and step 407 on the at least one pair of triangles, at least one ratio is obtained, and step 408 is performed according to the at least one ratio.

[0155] Step 408: The rotatable camera determines an orientation deviation between a predefined orientation corresponding to the predefined orientation image and a target orientation according to the at least one ratio obtained by determination.

[0156] Since there may be relatively discrete ratios in the at least one ratio, the specific implementation process of step 408 can be: perform linear clustering calculation on the at least one ratio to obtain the discrete value of each ratio; select a ratio whose discrete value is less than a preset discrete value from the at least one ratio; and determine the absolute value of the difference between the average value of the selected ratio and 1 as the orientation deviation between the predefined orientation corresponding to the predefined orientation image and the target orientation. The linear clustering calculation is performed on the at least one ratio, that is, the center point of the at least one ratio is determined by the K-means (hard clustering algorithm) clustering method, and the Euclidean distance from each ratio in the at least one ratio to the center point is determined as the discrete value of the ratio.

[0157] For example, the rotatable camera selects at least one discrete value less than a preset discrete value from the at least one ratio, and determines the average value of the selected ratio, which is marked as avg(λ). The difference between avg(λ) and 1 can be used to indicate the degree of deviation between the predefined orientation and the target orientation, that is, the closer avg(λ) is to 1, the closer the predefined orientation is to the target orientation; the more avg(λ) deviates from 1, the more the predefined orientation deviates from the target orientation. Therefore, the orientation deviation between the predefined orientation and the target orientation can be expressed by the following formula:

[0158] |avg(λ)-1|

[0159] It should be noted that, in the embodiment of the present invention, the rotatable camera may also directly determine the absolute value of the difference between the average value of the at least one ratio and 1 as the orientation deviation between the predefined orientation corresponding to the predefined orientation image and the target orientation.

[0160] When the rotatable camera performs the operations of steps 402 to 408 for each of the multiple predefined orientation images stored, for each of the multiple predefined orientations corresponding one-to-one to the multiple predefined orientation images, the rotatable camera can determine the orientation deviation between the target orientation and the predefined orientation, and obtain multiple orientation deviations corresponding one-to-one to the multiple predefined orientations.

[0161] Step 409: Select a predefined direction from a plurality of predefined directions, the direction deviation between which and the target direction is less than a preset direction deviation.

[0162] The preset azimuth deviation is a preset azimuth deviation. When the azimuth deviation between the target azimuth and the predefined azimuth is not less than the preset azimuth deviation, it indicates that the target azimuth and the predefined azimuth are far apart, and the rotatable camera does not perform any operation at this time; when the azimuth deviation between the target azimuth and the predefined azimuth is less than the preset azimuth deviation, it indicates that the target azimuth and the predefined azimuth are close, and the rotatable camera can rotate according to the preset azimuth deviation. Figure 3A In the embodiment shown in Table 1, the prompt information set for the predefined orientation is determined, and the prompt information set for the selected predefined orientation is determined as the prompt information that needs to be superimposed in the video image. Specifically, the rotatable camera can implement this process through the following steps 410 and 411.

[0163] Step 410: The rotatable camera determines the prompt information set for the selected predefined orientation as the prompt information that needs to be superimposed in the video picture.

[0164] After the rotatable camera determines the prompt information that needs to be superimposed on the video image, the prompt information can be superimposed on the video image in the following two possible implementation methods.

[0165] (1) The rotatable camera can superimpose prompt information set for the selected predefined orientation in the video image.

[0166] Specifically, the rotatable camera can superimpose the prompt information set for the selected predefined orientation in the current video picture through an on-screen display (OSD) mode, that is, after the rotatable camera determines the prompt information set for the predefined orientation, it determines the encoding information of the prompt information and creates an OSD environment, that is, initializes the OSD. The rotatable camera synchronizes and superimposes the encoding information of the prompt information and the frame data of the video picture, and writes the superimposed data into the created OSD environment through digital signal processing (DSP) technology to obtain the video picture after the superimposed prompt information.

[0167] Afterwards, the rotatable camera sends the video picture with the prompt information superimposed on it to the monitoring client, and the monitoring client displays the video picture with the prompt information superimposed on it. When the monitoring client receives the video picture with the prompt information superimposed on it sent by the rotatable camera, it directly displays the video picture with the prompt information superimposed on it. At this time, if the user is controlling the rotatable camera to rotate through the monitoring client, the user can view the prompt information in the currently displayed video picture to understand the approximate position of the current rotatable camera.

[0168] (2) The monitoring client superimposes prompt information set for the selected predefined position in the video image.

[0169] After the rotatable camera determines the prompt information set for the selected predefined orientation as the prompt information that needs to be superimposed in the video screen, the rotatable camera sends the prompt information and the identification of the video screen to the monitoring client. When the monitoring client receives the identification of the video screen, it determines the video screen from the received video screens according to the identification of the video screen, and then superimposes the prompt information on the video screen. The identification of the video screen can be the frame number of the video frame corresponding to the video screen sent by the rotatable camera.

[0170] The implementation process of the monitoring client superimposing the prompt information on the video screen may refer to the implementation process of the rotatable camera superimposing the prompt information on the video screen.

[0171] Optionally, when the corresponding relationship shown in Table 1 is stored in the monitoring client, after the rotatable camera determines the prompt information set for the selected predefined position as the prompt information that needs to be superimposed in the video screen, the rotatable camera sends the identifier of the selected predefined position and the identifier of the video screen to the monitoring client. When the monitoring client receives the identifier of the predefined position sent by the rotatable camera, it determines the prompt information set for the predefined position from the corresponding relationship shown in Table 1 according to the identifier of the predefined position, and determines the video screen from the received video screen according to the identifier of the video screen, and then superimposes the prompt information on the video screen.

[0172] After the monitoring client superimposes the prompt information on the video picture, the monitoring client may directly display the video picture after the superimposed prompt information.

[0173] In an embodiment of the present invention, a rotatable camera acquires a video image, and selects a predefined orientation whose orientation deviation from a target orientation is less than a preset orientation deviation from a plurality of predefined orientations corresponding one to one with a plurality of predefined orientation images; since the target orientation is the acquisition orientation of the video image, an orientation deviation less than the preset orientation deviation indicates that the target orientation is substantially close to the predefined orientation, and prompt information set for the selected predefined orientation can be superimposed on the video image at this time, without the need for the state parameters of the target orientation and the predefined orientation to be completely consistent before prompt information set for the predefined orientation can be superimposed on the video image, thereby improving the flexibility of determining prompt information in video surveillance.

[0174] When, according to the video screen and the plurality of predefined position images, the process of determining whether to determine the prompt information set for the predefined position as the prompt information that needs to be superimposed in the video screen is performed by Figure 1 When the monitoring server in the embodiment of the present invention is executed, another method for determining prompt information in video monitoring is provided, such as Figure 5 As shown, the method comprises the following steps:

[0175] Step 501: The monitoring server obtains the video image.

[0176] Specifically, when the rotatable camera is currently rotating, a video picture is collected in real time and sent to a monitoring server in the monitoring system. When the monitoring server receives the video picture collected in real time, the video picture collected in real time can be directly determined as the video picture. The monitoring server can also select a video picture from the video pictures received in the preset time period closest to the current time before the current time at every preset time period, and determine the selected video picture as the video picture.

[0177] Step 502: The monitoring server determines the orientation deviation between the target orientation and each of the plurality of predefined orientations based on the video image and a plurality of predefined orientation images stored in advance, wherein the plurality of predefined orientation images correspond to the plurality of predefined orientations one by one.

[0178] The implementation process of step 502 can refer to Figure 4A The implementation process of steps 402 to 407 in the embodiment of the present invention will not be elaborated in detail here.

[0179] Step 503: The monitoring server selects a predefined direction from the multiple predefined directions, the direction deviation between which and the target direction is less than a preset direction deviation.

[0180] The implementation process of step 503 and Figure 4A The implementation process of step 408 in is basically the same and will not be elaborated here.

[0181] When the monitoring server selects a predefined direction from the multiple predefined directions and the direction deviation of the target direction is less than the preset direction deviation, it indicates that the target direction is close to the predefined direction. Figure 3A In the embodiment shown in Table 1, the prompt information set for the predefined position is determined, and the prompt information set for the selected predefined position is determined as the prompt information to be superimposed in the video screen. Specifically, the monitoring server can implement this process through the following steps 504 and 505.

[0182] Step 504: The monitoring server determines the prompt information set for the selected predefined position as the prompt information that needs to be superimposed in the video picture.

[0183] After the monitoring server determines the prompt information that needs to be superimposed on the video screen, the prompt information can be superimposed on the video screen in the following two possible implementation methods.

[0184] (1) The monitoring server superimposes prompt information set for the selected predefined position in the video image.

[0185] The implementation process of the monitoring server superimposing the prompt information on the video screen can be referred to Figure 4A The implementation process of the rotatable camera in step 410 superimposing the prompt information on the video screen.

[0186] Afterwards, the monitoring server sends the video picture with the prompt information superimposed thereon to the monitoring client, and the monitoring client displays the video picture with the prompt information superimposed thereon.

[0187] (2) The monitoring client superimposes prompt information set for the selected predefined position in the video image.

[0188] Specifically, after the monitoring server determines the prompt information set for the selected predefined direction as the prompt information that needs to be superimposed on the video screen, the monitoring server sends the prompt information and the identifier of the video screen to the monitoring client, and the monitoring client superimposes the prompt information on the video screen and displays the video screen after the superimposed prompt information.

[0189] Optionally, when the corresponding relationship shown in Table 1 above is stored in the monitoring client, after the monitoring server determines the prompt information set for the selected predefined direction as the prompt information that needs to be superimposed on the video screen, the monitoring server sends the identifier of the selected predefined direction and the identifier of the video screen to the monitoring client, and the monitoring client still superimposes the prompt information on the video screen and displays the video screen after the superimposed prompt information.

[0190] In an embodiment of the present invention, a monitoring server acquires a video screen, and selects a predefined orientation whose orientation deviation from a target orientation is less than a preset orientation deviation from a plurality of predefined orientation images. Since the target orientation is the acquisition orientation of the video screen, and the orientation deviation is less than the preset orientation deviation, it can be indicated that the target orientation is basically close to the predefined orientation. At this time, prompt information set for the selected predefined orientation can be superimposed on the video screen, without the need to superimpose prompt information set for the predefined orientation on the video screen only when the state parameters of the target orientation and the predefined orientation are completely consistent, thereby improving the flexibility of determining prompt information in video monitoring.

[0191] In the present application, in addition to providing the method for determining prompt information in video surveillance described in the above embodiments, a device for determining prompt information in video surveillance is also provided. The following embodiments are used to describe in detail the device for determining prompt information in video surveillance provided in the embodiments of the present invention.

[0192] See also Fig. 6A The embodiment of the present invention provides a device 600 for determining prompt information in video surveillance. The device 600 includes an acquisition module 601, a first determination module 602, a selection module 603 and a second determination module 604.

[0193] Acquisition module 601, used to execute Figure 4A Step 401 in the embodiment or Figure 5 Step 501 in the embodiment.

[0194] The first determination module 602 is used to determine the orientation deviation between the target orientation and each of the multiple predefined orientations based on the video image and multiple predefined orientation images stored in advance, and the multiple predefined orientation images correspond to the multiple predefined orientations one by one.

[0195] Selection module 603 is used to execute Figure 4A Step 409 in the embodiment or Figure 5 Step 503 in the embodiment.

[0196] The second determining module 604 is used to execute Figure 4A Step 410 in the embodiment or Figure 5 Step 504 in an embodiment.

[0197] Optionally, the first determining module 602 includes:

[0198] Get unit for execution Figure 4A Step 402 in the embodiment;

[0199] A determination unit is used to determine, when the number of the multiple pairs of local feature points is greater than a preset number, an orientation deviation between a predefined orientation corresponding to the predefined orientation image and a target orientation based on the multiple pairs of local feature points.

[0200] Optionally, the predefined orientation image is an image including first-order edge features;

[0201] The acquisition unit is specifically used for:

[0202] Determine the background image of the video screen by using a Gaussian mixture background modeling method;

[0203] Processing the background image of the video screen to obtain a processed image, wherein the processed image is an image including a first-order edge feature;

[0204] The feature points in the processed image and the feature points in the predefined orientation image are matched by a scale-invariant feature transform (SIFT) algorithm to obtain a plurality of matched pairs of local feature points.

[0205] Optionally, the determining unit includes:

[0206] Select subunit for execution Figure 4A Step 403 in the embodiment;

[0207] Build subunits for execution Figure 4A In step 404 of the embodiment, the length of each side of the first triangle and the second triangle is greater than a preset side length, and each internal angle of the first triangle and the second triangle is greater than a preset angle.

[0208] The first determining subunit is used to execute Figure 4A Step 405 in the embodiment;

[0209] The spatial transformation subunit is used to perform Figure 4A Step 406 in the embodiment;

[0210] The second determining subunit is used to execute Figure 4A Step 407 in the embodiment;

[0211] The third determining subunit is used to execute Figure 4A Step 408 in an embodiment.

[0212] Optionally, the space transformation subunit is specifically used to:

[0213] Performing an affine transformation on the first triangle so that the sum of the squares of the differences between the inner angles corresponding to the matched local feature points in the affine-transformed triangle and the second triangle is less than a preset value;

[0214] Determine the average value of the ratios of the corresponding side lengths of the affine transformed triangle and the second triangle;

[0215] When the average value is greater than 1, the triangle after affine transformation is reduced by taking the average value as the reduction ratio to obtain the triangle after spatial transformation;

[0216] When the average value is not greater than 1, the triangle after affine transformation is enlarged with the inverse of the average value as the enlargement ratio to obtain the triangle after spatial transformation.

[0217] Optionally, the second determining subunit is specifically configured to:

[0218] Determine the distance between the center of the spatially transformed triangle and the center of the video screen where the spatially transformed triangle is located to obtain a first distance;

[0219] Determine the distance between the center of the second triangle and the center of the image where the second triangle is located to obtain a second distance;

[0220] The ratio between the first distance and the second distance is determined as the ratio between the field of view deflection angle of the spatially transformed triangle and the field of view deflection angle of the second triangle.

[0221] Optionally, the third determining subunit is specifically configured to:

[0222] Performing linear clustering calculation on the at least one ratio to obtain a discrete value of each ratio;

[0223] Selecting a ratio whose discrete value is smaller than a preset discrete value from the at least one ratio;

[0224] The absolute value of the difference between the average of the selected ratios and 1 is determined as the orientation deviation between the predefined orientation corresponding to the predefined orientation image and the target orientation.

[0225] Alternatively, see Figure 6B The device 600 also includes a setting module 605, a third determination module 606, a processing module 607 and a storage module 608.

[0226] Setting module 605, for executing Figure 3A Step 301 in the embodiment;

[0227] The third determining module 606 is used to execute Figure 3A Step 302 in the embodiment;

[0228] Processing module 607, used to execute Figure 3A Step 303 in the embodiment;

[0229] Storage module 608, used to execute Figure 3A Step 304 in an embodiment.

[0230] In an embodiment of the present invention, a video screen is acquired, and a predefined orientation whose orientation deviation from the target orientation is less than a preset orientation deviation is selected from a plurality of predefined orientations corresponding to a plurality of predefined orientation images. Since the target orientation is the acquisition orientation of the video screen, the orientation deviation being less than the preset orientation deviation can indicate that the target orientation and the predefined orientation are basically close. At this time, the prompt information set for the selected predefined orientation can be determined as the prompt information that needs to be superimposed in the video screen. The prompt information set for the selected predefined orientation does not need to be determined as the prompt information that needs to be superimposed in the video screen only when the state parameters of the target orientation and the predefined orientation are completely consistent, thereby improving the flexibility of determining prompt information in video monitoring.

[0231] It should be noted that: the information display device in video surveillance provided by the above embodiment only uses the division of the above functional modules as an example when displaying information in video surveillance. In actual applications, the above functions can be assigned to different functional modules as needed to complete all or part of the functions described above. In addition, the information display device in video surveillance provided by the above embodiment and the information display method embodiment in video surveillance belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0232] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (for example: coaxial cable, optical fiber, data subscriber line (Digital Subscriber Line, DSL)) or wireless (for example: infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0233] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0234] The above-mentioned embodiments are provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining prompt information in video surveillance, characterized in that: The method comprises: In the process of finding the location of the target object, a video image is obtained; Based on the video picture and a plurality of pre-stored predefined directions, a predefined direction close to a target direction is selected from the plurality of predefined directions, wherein the target direction is a collection direction of the video picture; The prompt information set for the selected predefined orientation is determined as the prompt information of the video picture of the target orientation.

2. The method according to claim 1, characterized in that Selecting a predefined orientation close to the target orientation from the plurality of predefined orientations specifically includes: According to the video picture and the plurality of predefined orientation images, it is determined whether to determine the prompt information set for the predefined orientation as the prompt information of the video picture of the target orientation.

3. The method according to claim 1 or 2, characterized in that The predefined positions close to the target position are specifically: A predefined direction whose direction deviation from the target direction is less than a preset direction deviation.

4. The method according to claim 1, characterized in that To obtain the video screen: Get the video footage shot by the PTZ camera.

5. The method according to claim 1, characterized in that Also includes: The prompt information set for the selected predefined orientation is superimposed on the video screen.

6. A device for determining prompt information in video surveillance, characterized in that: The device comprises: In the process of finding the location of the target object, the acquisition module is used to acquire the video picture; A first determining module, configured to determine a plurality of pre-stored predefined locations; A selection module, configured to select a predefined orientation close to a target orientation from the plurality of predefined orientations based on the video image, wherein the target orientation is a collection orientation of the video image; The second determining module is used to determine the prompt information set for the selected predefined orientation as the prompt information of the video picture of the target orientation.

7. The device for determining prompt information in video surveillance according to claim 6, wherein: The selection module is specifically used for: According to the video picture and the plurality of predefined orientation images, it is determined whether to determine the prompt information set for the predefined orientation as the prompt information of the video picture of the target orientation.

8. The device for determining prompt information in video surveillance according to claim 6 or 7, wherein: The predefined positions close to the target position are specifically: A predefined direction whose direction deviation from the target direction is less than a preset direction deviation.

9. The device for determining prompt information in video surveillance according to claim 6, characterized in that: The target object is a video image captured by a pan-tilt RTZ camera.

10. An information display device for video surveillance, the structure of the information display device for video surveillance comprising a processor and a memory, wherein: The memory is used to store programs; The processor is configured to execute the program stored in the memory to perform: In the process of finding the location of the target object, a video image is obtained; Based on the video picture and a plurality of pre-stored predefined directions, a predefined direction close to a target direction is selected from the plurality of predefined directions, wherein the target direction is a collection direction of the video picture; The prompt information set for the selected predefined orientation is determined as the prompt information of the video picture of the target orientation.

Images