Method, device, equipment and computer storage medium for monitoring target tracking and display

Abstract

The monitoring target tracking display method, device, equipment and computer storage medium provided by the embodiments of the present invention can avoid the failure of tracking caused by the action of the monitoring target resulting in the monitoring image taken from a single angle not being able to obtain effective information by displaying the monitoring images taken from multiple angles at the same shooting time in the form of a three-dimensional model, and facilitate the user to establish a three-dimensional perception of the flat monitoring image, so as to better observe the monitoring target. In addition, by identifying the monitoring target to generate the first monitoring information and displaying it in the second area of ​​the first display interface, it is convenient for the user to obtain the information of the monitoring target with a summary nature, so as to better observe the monitoring target.

Description

Technical Field

[0001] The present invention relates to the field of monitoring security technology, and in particular to a method, device, equipment and computer storage medium for monitoring target tracking and display. Background Art

[0002] In security scenarios, surveillance images or videos captured by surveillance video are viewed to track the movements of surveillance targets. In order to improve the tracking efficiency of surveillance targets, surveillance images and other surveillance information of surveillance targets can be archived for easy viewing. However, in actual practice, surveillance images often block part of the body image due to the walking posture of the surveillance target, resulting in unsatisfactory image feature extraction of the surveillance target due to single-angle image processing of the surveillance target, which affects the observation and tracking of the surveillance target. Summary of the invention

[0003] The embodiments of the present invention provide a method, device, equipment and computer storage medium for tracking and displaying a monitoring target, so as to solve the problem in the prior art that single-angle image processing of the monitoring target has poor observation and tracking effects on the monitoring target.

[0004] An embodiment of the present invention provides a method for monitoring target tracking and display, comprising:

[0005] Obtain surveillance images taken by a multi-view surround camera group from N angles at the same shooting time, where N is a positive integer greater than 2;

[0006] Establishing a straight N-prism stereo model, and displaying the straight N-prism stereo model on a first area of ​​a first display interface, wherein each side of the straight N-prism stereo model displays a monitoring image;

[0007] The monitoring target is identified on the monitoring image, first monitoring information of the monitoring target is generated according to the identification result, and the first monitoring information is displayed on the second area of ​​the first display interface.

[0008] Optionally, the method further comprises at least one of the following steps:

[0009] In response to the first model display adjustment instruction, the straight N-prism stereo model is rotated by a specified angle with the height line of the straight N-prism stereo model as a rotation axis;

[0010] In response to the second model display adjustment instruction, the straight N-prism three-dimensional model is rotated by the angle indicated by the second model display adjustment instruction, with the axis indicated by the second model display adjustment instruction as the rotation axis.

[0011] Optionally, each side of the straight N-prism three-dimensional model displays a surveillance image obtained by cutting the captured surveillance image into a preset shape.

[0012] Optionally, performing surveillance target recognition on the surveillance image includes:

[0013] Identifying a monitoring target image region where a monitoring target is located in the monitoring image, and a face image region where a face of the monitoring target is located;

[0014] The method further comprises:

[0015] On the monitoring image of the side of the straight N-prism stereo model, display an animation marking the monitoring target image area and the face image area;

[0016] The monitoring target image area and the face image area are extracted to obtain the second monitoring information and determine the monitoring target corresponding to the second monitoring information, the second monitoring information is moved to a specified position floating outside the straight N-prism model for display, and a transition animation of the movement is displayed.

[0017] Optionally, generating first monitoring information of the monitoring target according to the recognition result, and displaying the first monitoring information on the second area of ​​the first display interface includes:

[0018] saving the second monitoring information;

[0019] Determine whether the first monitoring information of the monitoring target exists in the second area;

[0020] If the first monitoring information of the monitoring target does not exist in the second area, generating the first monitoring information of the monitoring target according to the second monitoring information, and displaying it at the reserved position in the second area, and generating a new reserved position;

[0021] Displaying a transition animation of moving the second monitoring information of the monitoring target to the position of the first monitoring information of the monitoring target;

[0022] The reserved position is the starting position or the ending position of the arrangement.

[0023] Optionally, if the first monitoring information of the monitoring target exists in the second area, the method further includes:

[0024] The first monitoring information of the monitoring target is updated according to the second monitoring information and then displayed.

[0025] Optionally, the first monitoring information of the monitoring target includes at least one of the following:

[0026] The monitoring target area and / or the face image area, wherein the monitoring target area and the face image area belong to the second monitoring information;

[0027] Removing a background image or extracting a human body mask from the monitoring target area of ​​the second monitoring information to obtain a gait diagram of the monitoring target;

[0028] A three-dimensional model of the monitoring target is obtained by modeling the monitoring target using the gait diagram of the monitoring target.

[0029] Optionally, the three-dimensional model is a dynamic three-dimensional model that is modeled using a gait graph corresponding to at least one shooting moment of the monitored target and displays the walking movements of the monitored target.

[0030] Optionally, if the first monitoring information includes a three-dimensional model of the monitoring target, the method further includes:

[0031] In response to a first display operation instruction for the three-dimensional model, a display viewing angle state of the three-dimensional model is switched, wherein the display viewing angle state includes a dynamic viewing angle state and / or at least one fixed viewing angle state.

[0032] Optionally, if the first monitoring information includes a dynamic three-dimensional model of the monitoring target, the method further includes:

[0033] In response to a second display operation instruction for the three-dimensional model, the action state of the dynamic three-dimensional model is switched, and the action state includes a walking action state, a paused walking action state, and a standing state.

[0034] Optionally, the method further comprises:

[0035] In response to the instruction to view detailed information, at least part of the first monitoring information corresponding to the instruction to view detailed information is magnified and displayed in a designated area.

[0036] Based on the same inventive concept, an embodiment of the present invention further provides a device for monitoring target tracking and display, comprising:

[0037] A monitoring image acquisition module is used to acquire monitoring images taken by a multi-view surround camera group from N angles at the same shooting time, where N is a positive integer greater than 2;

[0038] A monitoring image display module, used for establishing a straight N-prism stereo model and displaying the straight N-prism stereo model on a first area of ​​a first display interface, wherein each side of the straight N-prism stereo model displays a monitoring image;

[0039] The monitoring information display module is used to identify the monitoring target of the monitoring image, generate first monitoring information of the monitoring target according to the identification result, and display the first monitoring information in the second area of ​​the first display interface.

[0040] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor;

[0041] Wherein, the processor is configured to execute the instructions to implement the method of monitoring target tracking and display.

[0042] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is used to implement any one of the monitoring target tracking and display methods.

[0043] The beneficial effects of the present invention are as follows:

[0044] The monitoring target tracking display method, device, equipment and computer storage medium provided by the embodiments of the present invention can avoid the failure of tracking due to the action of the monitoring target causing the monitoring image taken from a single angle to obtain effective information (for example, the monitoring target waves his arms while walking, causing his face to be blocked in the monitoring image taken from the side), by displaying the monitoring images taken from multiple angles by the multi-view surround camera group at the same shooting time in the form of a three-dimensional model, and facilitate the user to establish a three-dimensional feeling for the flat monitoring image, so as to better observe the monitoring target. In addition, by identifying the monitoring target to generate the first monitoring information and displaying it in the second area of ​​the first display interface, it is convenient for the user to obtain the information of the monitoring target with summary nature, so as to better observe the monitoring target. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of a method for tracking and displaying a monitoring target provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a first page of a method for implementing monitoring target tracking and display provided by an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of the position arrangement of a multi-view surround camera group in an embodiment of the present invention;

[0048] Figure 4 Schematic diagram of the structure of an image acquisition device in an embodiment of the present invention;

[0049] Figure 5 Schematic diagram of a straight N-prism stereo model in an embodiment of the present invention;

[0050] Figure 6 It is a schematic diagram of the display effect change of the straight N-prism three-dimensional model in response to the first model display adjustment instruction in an embodiment of the present invention;

[0051] Figure 7 It is a schematic diagram of the display effect change of the straight N-prism three-dimensional model in response to the second model display adjustment instruction in an embodiment of the present invention;

[0052] Figure 8 A schematic diagram showing the animation effect of marking the monitoring target image area and the face image area in an embodiment of the present invention;

[0053] Fig. 9 A flowchart of some steps of a method for tracking and displaying a monitoring target provided by an embodiment of the present invention;

[0054] Fig.10 A gait diagram of a monitoring target in an embodiment of the present invention;

[0055] Fig.11 This is one of the schematic diagrams of the first monitoring information in an embodiment of the present invention;

[0056] Fig.12 This is a second schematic diagram of the first monitoring information in an embodiment of the present invention;

[0057] Fig.13 This is a schematic diagram of the animation effect of the first page in an embodiment of the present invention;

[0058] Fig.14 It is a schematic diagram of the hourly effect of the dynamic three-dimensional model of the monitoring target in an embodiment of the present invention;

[0059] Fig.15 A schematic diagram of a second page of a method for implementing monitoring target tracking and display provided by an embodiment of the present invention;

[0060] Fig.16 A schematic diagram of the structure of a device for tracking and displaying a monitoring target provided by an embodiment of the present invention;

[0061] Fig.17 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described below with reference to the accompanying drawings and examples. However, the exemplary embodiments can be implemented in various forms and should not be understood as being limited to the embodiments described herein; on the contrary, these embodiments are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the exemplary embodiments to those skilled in the art. The same figure marks in the figures represent the same or similar structures, and thus their repeated descriptions will be omitted. The words expressing position and direction described in the present invention are all explained using the accompanying drawings as examples, but changes can be made as needed, and the changes made are all included in the scope of protection of the present invention. The drawings of the present invention are only used to illustrate the relative position relationship and do not represent the true proportions.

[0063] It should be noted that specific details are described in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in a variety of other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The subsequent description of the specification is a preferred embodiment of the present application, but the description is for the purpose of illustrating the general principles of the present application and is not intended to limit the scope of the present application. The scope of protection of the present application shall be determined by the definition of the attached claims.

[0064] The following is a detailed description of the monitoring target tracking and display method, device, electronic device and computer readable storage medium provided by the embodiments of the present invention in conjunction with the accompanying drawings. It should be noted that the monitoring target involved in the embodiments of the present invention can be a moving target such as a pedestrian or a vehicle.

[0065] The embodiment of the present invention provides a method for monitoring target tracking display, such as Figure 1 and Figure 2 As shown, Figure 1 A flow chart showing a method for monitoring target tracking display is shown. Figure 2 The first page of the method for implementing the monitoring target tracking display is illustrated. Specifically, the monitoring target tracking display method includes:

[0066] S110. Obtain surveillance images captured by a multi-view surround camera group from N angles at the same shooting time, where N is a positive integer greater than 2.

[0067] In a specific implementation process, the multi-view surround camera group can be as follows: Figure 3The arrangement is shown in the figure, that is, multiple cameras set at different positions monitor the same position, and capture surveillance images of the surveillance target from different angles at the same shooting time. The method of acquiring the surveillance image in step S110 can be to directly capture the surveillance image by the multi-view surround camera group, or to extract the frame image of the surveillance video after recording the surveillance video by the multi-view surround camera group as the surveillance image.

[0068] Furthermore, the multi-view surround camera group can be further improved as follows Figure 4 The image acquisition device shown in the figure: the image acquisition device includes a main control module 101, a deserialization module 102, and multiple image acquisition modules 103, each of which includes a camera 104 and a serialization unit 105 connected to the camera. The main control module 101 is connected to the deserialization module 102, and the deserialization module 102 is connected to each serialization unit 105. The camera 104 is used to collect image sensor signals, process them into multiple image frame signals according to the protocol, and transmit them to the connected serialization unit 105; each serialization unit 105 is used to serially encode the multiple image frame signals into a single high-frequency signal, and transmit the single high-frequency signal to the deserialization module 102; the deserialization module 102 is used to parallelly process the single high-frequency signals received from multiple serialization units 105: deserialize the single high-frequency signals into corresponding multiple image frame signals, and transmit the obtained multiple image frame signals to the main control module 101 in parallel.

[0069] Since the above-mentioned multi-channel image frame signals are usually four channels, after the serialization unit 105 serially encodes the four-channel image frame signals into a single-channel signal, it can support longer-distance transmission. Therefore, in actual deployment, each image acquisition module and the deserialization module can be spaced at a longer distance. In this way, the single-channel high-frequency signal obtained after serialization modulation can support long-distance transmission of 15 to 20 meters, so that each camera can be deployed at a farther distance.

[0070] Furthermore, the image acquisition device further includes a reference clock and a synchronization signal generator, wherein:

[0071] The synchronization signal generator generates a synchronization control signal based on the clock signal of the reference clock;

[0072] The deserialization module 102 is also used to transmit the synchronization control signal to each serialization unit 105;

[0073] The serialization unit 105 is further configured to transmit the received synchronization control signal to the camera 104 to control the exposure of the camera.

[0074] In this way, it can be ensured that each camera can capture the monitoring image at the same shooting time, avoiding inaccurate time corresponding to the monitoring image due to signal delay.

[0075] S120: Create a straight N-prism 3D model, and display the straight N-prism 3D model in a first area of ​​a first display interface, wherein each side of the straight N-prism 3D model displays a monitoring image.

[0076] In the specific implementation process, the step S120 can be implemented by CSS3 Preserve-3D technology. In order to facilitate users to observe more monitoring images displayed on the side at the same time, such as Figure 2 , Figure 5 As shown, the bottom and top surfaces of the straight N-prism stereoscopic model are transparent. In order to further facilitate user observation, the straight N-prism stereoscopic model can be a regular N-prism stereoscopic model.

[0077] S130: performing surveillance target recognition on the surveillance image.

[0078] S140: Generate first monitoring information of the monitoring target according to the recognition result.

[0079] S150: Display the first monitoring information in a second area of ​​the first display interface.

[0080] In a specific implementation process, the monitoring image for monitoring target identification in step S130 may be the monitoring image displayed on the side of the straight N-prism stereoscopic model in step S120, or may be the monitoring image obtained in step S110. In order to realize the front-end and back-end separation processing of data, step S130 may be set as a back-end processing process, and the monitoring image for monitoring target identification in this process is the monitoring image obtained in step S110.

[0081] If the architecture of separating the front-end information display from the back-end monitoring information processing is adopted, then in the step S120, when the straight N-prism stereo model is established and the monitoring image is displayed on its side, considering that in order to allow the user to observe more monitoring images at the same time, the straight N-prism stereo model can be displayed at an inclined perspective, so that for the user, the display effect of each non-frontal side of the straight N-prism seen in this state is not a rectangle but an oblique parallelogram. Considering that the monitoring image may be shot at an angle facing the monitoring target, the display effect of the rectangular monitoring image when the straight N-prism stereo model is displayed at an inclined angle is a distorted oblique parallelogram, and the monitoring target in the monitoring image is also distorted into an inclined display state. In order to allow the user to see the monitoring target when observing the straight N-prism stereo model, it is further optional that each side of the straight N-prism stereo model displays a monitoring image after the captured monitoring image is cut into a preset shape.

[0082] For example, for a right quadrangular prism stereoscopic model, when the two side faces and the top face can be seen at the same time, the display effect of the side face shape is an oblique parallelogram. If the original monitoring images corresponding to the two side faces are cropped (for example, the image areas of the two opposite corners of the rectangular original monitoring image are cropped to make the remaining monitoring image area an oblique parallelogram), then when the cropped monitoring image is displayed on the side face of the right quadrangular prism, the monitoring target in the monitoring image will be displayed to the user at a positive angle. In the specific implementation process, the above-mentioned cropping process can be realized by CSS3 clip-path technology.

[0083] In this way, by displaying the surveillance images taken from multiple angles by the multi-view surround camera group at the same shooting time in the form of a three-dimensional model, it is possible to avoid the failure of tracking caused by the action of the surveillance target resulting in the inability to obtain effective information from the surveillance image taken from a single angle (for example, the surveillance target waves his arms while walking, resulting in his face being blocked in the surveillance image taken from the side), and it is convenient for users to establish a three-dimensional feeling for the flat surveillance image, so as to better observe the surveillance target. In addition, by identifying the surveillance target to generate the first surveillance information and displaying it in the second area of ​​the first display interface, it is convenient for users to obtain information of the surveillance target with a summary nature, so as to better observe the surveillance target.

[0084] After the straight N-prism stereo model is displayed on the first area of ​​the first display interface, in order to facilitate the user to observe the monitoring images on different sides of the straight N-prism stereo model, the viewing angle of the straight N-prism stereo model can also be adjusted according to the user's adjustment instructions.

[0085] Optionally, the method further comprises at least one of the following steps:

[0086] S121 . In response to a first model display adjustment instruction, the straight N-prism 3D model is rotated by a specified angle with the height line of the straight N-prism 3D model as a rotation axis.

[0087] For example, Figure 6 As shown, the straight N-prism stereo model is a regular quadrangular prism stereo model, then a left-rotating touch component and a right-rotating touch component can be set at the designated position of the first area respectively. When the user clicks the touch component once (for example, the "right-rotating" touch component) with the mouse to trigger the first model display adjustment instruction, the regular quadrangular prism stereo model will rotate right 90° with the height line passing through the center of the bottom surface and the center of the top surface as the rotation axis to change the display viewing angle.

[0088] S122. In response to a second model display adjustment instruction, using the axis indicated by the second model display adjustment instruction as a rotation axis, rotating the straight N-prism three-dimensional model by an angle indicated by the second model display adjustment instruction.

[0089] For example, a draggable touch area can be set at a specified position of the first area or the entire first area. When the user drags the mouse in the touch area, the rotation axis and rotation angle are determined according to the mouse dragging trajectory as the second model display adjustment instruction to rotate the straight N prism stereo model, so as to achieve the following. Figure 7 The straight N-prism stereo model shown demonstrates the transformation of viewing angle.

[0090] Of course, the first model display adjustment instruction and the second model display adjustment instruction may also be instructions in other forms, such as instructions in the form of control codes input by a user, which are not specifically limited here.

[0091] In this way, the display viewing angle of the straight N-prism three-dimensional model is changed through the model display adjustment instruction, which facilitates the user to observe the monitoring image.

[0092] In the case where the monitoring target is a pedestrian, optionally, in step S130, the monitoring target recognition is performed on the monitoring image, including:

[0093] Identify a monitoring target image region where a monitoring target is located in the monitoring image, and a face image region where a face of the monitoring target is located.

[0094] The method further comprises:

[0095] S131 . Displaying an animation marking the monitoring target image area and the face image area on the monitoring image on the side of the straight N-prism stereoscopic model.

[0096] For example, Figure 8 As shown, on the side of the straight N-prism stereo model, the identified monitoring target image and the facial image area are marked with an image frame and an image frame drawing and highlighting animation are displayed, or on the side of the straight N-prism stereo model, the identified monitoring target image and the facial image area are displayed with an outward floating animation. It can also be other forms of animation used to highlight the monitoring target image and the facial image area to the user, which is not specifically limited here.

[0097] S132: extract the monitoring target image area and the face image area to obtain second monitoring information, and determine the monitoring target corresponding to the second monitoring information.

[0098] S133: Move the second monitoring information to a designated position floating outside the straight N-prism model for display, and display a transition animation of the movement.

[0099] In a specific implementation process, when the face image region where the face of the monitoring target is located is identified, only a designated monitoring image (eg, Figure 8 As shown, the surveillance image taken by the camera of the specified perspective in the multi-perspective surround camera group) is identified, and the facial image area identified for this surveillance image is extracted in step S132 to obtain the second surveillance information; it is also possible to identify a random surveillance image among the multiple surveillance images, and extract the facial image area identified for this surveillance image in step S132 to obtain the second surveillance information; it is also possible to identify all the surveillance images among the multiple surveillance images, and extract the facial image area with the highest image score (for example, with clarity as the scoring standard) in step S132 to obtain the second surveillance information; no specific limitation is made here. The specific floating display position of the second surveillance information can be as follows Figure 8 As shown, it is located at the center of the view of the straight N-prism stereo model, wherein the face image area in the second monitoring information is located at the very center, and the monitoring target image areas corresponding to the monitoring information on each side surround the face image area, so as to more intuitively show the user the monitoring target targeted at the current image shooting moment.

[0100] If multiple monitoring targets appear simultaneously in the monitoring image taken at the current image shooting moment, the corresponding monitoring target image area and face image area will be extracted for each monitoring target in step S132 to obtain the corresponding second monitoring information, and the second monitoring information of each monitoring target will be displayed simultaneously or sequentially at the designated position outside the straight N-prism model in step S133.

[0101] In order to reduce the computing power consumed by step S130 as much as possible, optionally, as Fig. 9 As shown, in the step S130, identifying the monitoring target image area where the monitoring target in the monitoring image is located specifically includes:

[0102] S201: Perform surveillance target recognition on a first surveillance image to determine a surveillance target image area in the first surveillance image and first position information of the surveillance target image area.

[0103] In a specific implementation process, the location information includes the vertex coordinates, length and width of the monitoring target image area. The first monitoring image is a monitoring image of the multi-view surround camera group among the N monitoring images, and can be a specified monitoring image among the multiple monitoring images (for example, Figure 8 As shown, the monitoring image taken by the camera of the specified viewing angle in the multi-view surround camera group can also be a random monitoring image among the N monitoring images.

[0104] S202: Input the first position information into a position relationship mapping model to obtain second position information of the monitoring target in each second monitoring image output by the position relationship mapping model, wherein the second monitoring image is the other monitoring image among the N monitoring images except the first monitoring image.

[0105] S203: do not repeatedly select a second monitoring image.

[0106] If the step S203 is selected successfully, step S204 is executed; if all second monitoring images have been selected, subsequent steps are executed, such as the step of selecting the facial image area where the face of the monitoring target is located.

[0107] S204: Determine a monitoring target image area of ​​the monitoring target in the second monitoring image based on second position information of the monitoring target in the second monitoring image.

[0108] The above-mentioned position relationship mapping model can be a deep neural network (DNN) model or other machine learning models, which is not limited here. The position relationship mapping model can be trained by using multi-view monitoring images with pre-annotated monitoring target image areas as training samples. Since the specific training process is not an important part of the present invention, it is not described in detail here. Optionally, the step S140, generating the first monitoring information of the monitoring target according to the recognition result, includes:

[0109] S141, saving the second monitoring information;

[0110] S142: Determine whether the first monitoring information of the monitoring target exists in the second area.

[0111] If the result of step S142 is no, step S143 is executed.

[0112] S143: Generate first monitoring information of the monitoring target according to the second monitoring information.

[0113] The first monitoring information of the monitoring target includes at least one of the following:

[0114] (1) The monitoring target area and / or the face image area, wherein the monitoring target area and the face image area belong to the second monitoring information.

[0115] (2) removing the background image or extracting the human body mask from the monitoring target area of ​​the second monitoring information to obtain a gait diagram of the monitoring target. Fig.10 It illustrates the extraction of a human mask from the monitoring target area of ​​the second monitoring information to obtain a gait diagram of the monitoring target)

[0116] In a specific implementation process, for each shooting moment, the multi-view surround camera group will obtain multiple monitoring images, so the gait diagram at this shooting moment can be specified. The background image can be removed from the monitoring target area of ​​a specified monitoring image among the multiple monitoring images, or the human body mask can be extracted to obtain the gait diagram; the background image can be removed from the monitoring target area of ​​a random monitoring image among the multiple monitoring images, or the human body mask can be extracted to obtain the gait diagram; the background image can also be removed from the monitoring target area of ​​all monitoring images among the multiple monitoring images, or the human body mask can be extracted to obtain multiple gait diagrams, and then these gait diagrams are scored according to clarity, the proportion of the area of ​​the monitoring target area, etc., and the gait diagram with the highest score is used as the gait diagram at this shooting moment, and other gait diagrams at this shooting moment are discarded; no specific limitation is made here.

[0117] (3) A three-dimensional model of the monitoring target is obtained by modeling the monitoring target using the gait diagram of the monitoring target.

[0118] In the specific implementation process, the method of modeling the monitoring target using the gait diagram of the monitoring target can be implemented using existing technology (for example, the three-dimensional human body reconstruction method provided by Chinese patent CN106204718A, or other existing technology known to technical personnel in this field), which will not be described in detail here.

[0119] During the specific implementation process, the three-dimensional model of the monitored target can be a static three-dimensional model that is modeled using a gait diagram corresponding to at least one shooting moment of the monitored target and cannot switch the walking action state of the monitored target; it can also be a dynamic three-dimensional model that is modeled using a gait diagram corresponding to at least one shooting moment of the monitored target and displays the walking action of the monitored target.

[0120] Specifically, Figure 2 The first monitoring information shown in FIG. Fig.11 As shown, including the second monitoring information, the gait diagram and the shooting information of the monitoring image; or Figure 2 The first monitoring information shown in FIG. Fig.12 As shown, it includes a three-dimensional model of the monitored target, a gait diagram and shooting information of the monitored image.

[0121] It should be noted that the three-dimensional model of the monitoring target may be independently used as the first monitoring information, while the gait diagram of the monitoring target used to model the three-dimensional model may not be displayed as the first monitoring information in the second area.

[0122] In addition, the first monitoring information of the monitoring target may also include shooting information of the monitoring image, such as image shooting time, image resolution, exposure time, focal length, white balance and other parameters.

[0123] The step S150 of displaying the first monitoring information on the second area of ​​the first display interface includes:

[0124] S151: Display the first monitoring information at a reserved position in the second area. The reserved position is the starting position or the end position of the arrangement (e.g. Fig.13 shown).

[0125] S152: Generate a new reserved position.

[0126] In a specific implementation process, the process of generating a new reserved position may be displayed to the user or may not be displayed.

[0127] During the specific implementation process, when the multi-view surround camera group continuously captures surveillance images over a period of time and generates first surveillance information for each captured surveillance target, due to the size limitation of the first page, the second area may not be able to display all generated first surveillance information at the same time. At this time, the arrangement of the first surveillance information in the second area will automatically slide up and down to give priority to displaying the latest generated first surveillance information.

[0128] S153, displaying a transition animation (such as Fig.13 shown).

[0129] In this way, whenever the multi-view surround camera group captures a new monitoring target, a first monitoring information is generated for this monitoring target in the second area for display, and the latest generated first monitoring information is continuously placed in the order of generation time at the starting position or the end position of the arrangement of the first monitoring information in the second area, thereby being able to display to the user the time sequence in which different monitoring targets pass by the position of the multi-view surround camera group.

[0130] As an optional implementation, if the result of step S142 is yes, execute step S154.

[0131] S154: Update the first monitoring information of the monitoring target according to the second monitoring information and then display it.

[0132] In the specific implementation process, when the first monitoring information includes the second monitoring information, the second monitoring information corresponding to the current image shooting moment may replace the second monitoring information corresponding to the previous image shooting moment to update the first monitoring information; or for each monitoring target area in the first monitoring information, the monitoring target area at the current shooting moment and the monitoring target area at the previous shooting moment may be used to update and generate a video clip or dynamic image (for example, Graphics Interchange Format (GIF)) of the monitoring target area. When the first monitoring information includes the gait diagram, the gait diagram corresponding to the current image shooting moment may replace the gait diagram corresponding to the previous image shooting moment, or the gait diagram corresponding to the current image shooting moment and the gait diagram corresponding to the previous image shooting moment may be combined into a gait diagram sequence arranged in chronological order, or the gait diagram corresponding to the current image shooting moment and the gait diagram corresponding to the previous image shooting moment may be combined into a gait diagram video clip or a dynamic image may be generated. When the first monitoring information includes a dynamic three-dimensional model of the monitored target, the gait diagram sequence can be used to update the walking movements of the monitored target to show the user the characteristics of the walking movements of different monitored targets (for example, the gait of the monitored target is an inward-turned-feet gait, an outward-turned-feet gait, etc.).

[0133] In this way, by generating a first monitoring information for each monitoring target in the second area for display, and continuously updating the first monitoring information of the same monitoring target for display, all monitoring information of the same monitoring target can be integrated and archived for easy observation by users.

[0134] As another optional implementation, if the result of step S142 is yes, the first monitoring information of the monitoring target is not updated to reduce the amount of information to be displayed on the display interface and the amount of calculation occupied by changing the information to be displayed.

[0135] In the specific implementation process, the above two optional implementation methods can be implemented in combination, for example, the first monitoring information of the monitoring target is updated at a certain frequency (for example, every certain period of time), and no update is made at other times; the first monitoring information of the monitoring target can also be updated within a certain time after the first monitoring information of the monitoring target is generated, and after exceeding this time, the second monitoring information of the monitoring target at the shooting time outside this time will no longer be used to update the first monitoring information.

[0136] Further optionally, if the first monitoring information includes a three-dimensional model of the monitoring target, the method further includes:

[0137] S160. In response to a first display operation instruction for the three-dimensional model, switch a display viewing angle state of the three-dimensional model, where the display viewing angle state includes a dynamic viewing angle state and / or at least one fixed viewing angle state.

[0138] In the specific implementation process, when the three-dimensional model is displayed in step S150, the three-dimensional model can be displayed to the user in a dynamic perspective rotating around the central axis, or it can be displayed to the user from a static fixed perspective. Then, step S160 will change the display perspective state of the three-dimensional model displayed to the user. For example, Fig.14 As shown, the step S150 displays the three-dimensional model that rotates continuously to the user in a dynamic viewing state by default. When the user hovers the mouse pointer over the position of the three-dimensional model, or the user clicks a preset touch component (e.g. Fig.14 The user controls the mouse pointer to leave the position of the three-dimensional model, or the user clicks the preset touch component on the first page to trigger the first display operation instruction of the three-dimensional model, and the three-dimensional model is displayed to the user in a dynamic viewing state by rotating the three-dimensional model again.

[0139] Further optionally, if the first monitoring information includes a dynamic three-dimensional model of the monitoring target, the method further includes:

[0140] S170. In response to a second display operation instruction for the three-dimensional model, switching the action state of the dynamic three-dimensional model, the action state including a walking action state, a paused walking action state, and a standing at attention state.

[0141] In a specific implementation process, when the three-dimensional model is displayed in step S150, the dynamic three-dimensional model can be displayed to the user in a walking action state, a paused walking action state, or a standing state. Then, step S160 will change the action state of the dynamic three-dimensional model displayed to the user. For example, step S150 displays the dynamic three-dimensional model to the user in a walking action state by default. When the user hovers the mouse pointer over the position of the dynamic three-dimensional model, or the user clicks a preset touch component (such as Fig.14 The user controls the mouse pointer to leave the position of the dynamic three-dimensional model, or the user clicks the preset touch component on the first page to trigger the second display operation instruction of the three-dimensional model, and the action state of the three-dimensional model is switched to the walking action state for display to the user. When the user controls the mouse pointer to leave the position of the dynamic three-dimensional model, or the user clicks the preset touch component on the first page to trigger the second display operation instruction of the three-dimensional model again, the action state of the three-dimensional model is switched to the walking action state for display to the user.

[0142] Optionally, the method further comprises:

[0143] S180. In response to a view detailed information instruction, at least part of the first monitoring information corresponding to the view detailed information instruction is displayed in the first area, the second area, the third area, or the second display interface of the first display interface by enlarging the first area, the second area, the third area, or the second display interface.

[0144] In a specific implementation process, the designated area of ​​the designated page may be:

[0145] a first area of ​​the first display page;

[0146] a second area of ​​the first display page;

[0147] A third area on the first display page that does not belong to the first area and the second area;

[0148] A designated area in a second display page different from the first display page.

[0149] For example, Fig.15As shown, the first monitoring information includes the second monitoring information, the dynamic three-dimensional model, the gait diagram, and the shooting information of the monitoring image. When the user triggers the detailed information viewing instruction, the first monitoring information is magnified and displayed on another second display interface, so that the user can observe the first monitoring information more carefully.

[0150] Based on the same inventive concept, the embodiment of the present invention also provides a device for monitoring target tracking display, such as Fig.16 As shown, including:

[0151] The monitoring image acquisition module M110 is used to acquire monitoring images taken by the multi-view surround camera group from N angles at the same shooting time, where N is a positive integer greater than 2;

[0152] A monitoring image display module M120, configured to establish a straight N-prism stereo model and display the straight N-prism stereo model in a first area of ​​a first display interface, wherein each side of the straight N-prism stereo model displays a monitoring image;

[0153] The monitoring information display module M130 is used to identify the monitoring target of the monitoring image, generate first monitoring information of the monitoring target according to the identification result, and display the first monitoring information in the second area of ​​the first display interface.

[0154] Optionally, the monitoring image display module M120 is further used for:

[0155] In response to the first model display adjustment instruction, the straight N-prism stereo model is rotated by a specified angle with the height line of the straight N-prism stereo model as a rotation axis;

[0156] In response to the second model display adjustment instruction, the straight N-prism three-dimensional model is rotated by the angle indicated by the second model display adjustment instruction, with the axis indicated by the second model display adjustment instruction as the rotation axis.

[0157] Optionally, each side of the straight N-prism three-dimensional model displays a surveillance image obtained by cutting the captured surveillance image into a preset shape.

[0158] Optionally, performing surveillance target recognition on the surveillance image includes:

[0159] Identifying a monitoring target image region where a monitoring target is located in the monitoring image, and a face image region where a face of the monitoring target is located;

[0160] The monitoring image display module M120 is also used for:

[0161] On the monitoring image of the side of the straight N-prism stereo model, display an animation marking the monitoring target image area and the face image area;

[0162] The monitoring target image area and the face image area are extracted to obtain the second monitoring information and determine the monitoring target corresponding to the second monitoring information, the second monitoring information is moved to a specified position floating outside the straight N-prism model for display, and a transition animation of the movement is displayed.

[0163] Optionally, generating first monitoring information of the monitoring target according to the recognition result, and displaying the first monitoring information on the second area of ​​the first display interface includes:

[0164] saving the second monitoring information;

[0165] Determine whether the first monitoring information of the monitoring target exists in the second area;

[0166] If the first monitoring information of the monitoring target does not exist in the second area, generating the first monitoring information of the monitoring target according to the second monitoring information, and displaying it at the reserved position in the second area, and generating a new reserved position;

[0167] Displaying a transition animation of moving the second monitoring information of the monitoring target to the position of the first monitoring information of the monitoring target;

[0168] The reserved position is the starting position or the ending position of the arrangement.

[0169] Optionally, if the first monitoring information of the monitoring target exists in the second area, the monitoring information display module M130 is further used to:

[0170] The first monitoring information of the monitoring target is updated according to the second monitoring information and then displayed.

[0171] Optionally, the first monitoring information of the monitoring target includes at least one of the following:

[0172] The monitoring target area and / or the face image area, wherein the monitoring target area and the face image area belong to the second monitoring information;

[0173] Removing a background image or extracting a human body mask from the monitoring target area of ​​the second monitoring information to obtain a gait diagram of the monitoring target;

[0174] A three-dimensional model of the monitoring target is obtained by modeling the monitoring target using the gait diagram of the monitoring target.

[0175] Optionally, the three-dimensional model is a dynamic three-dimensional model that is modeled using a gait graph corresponding to at least one shooting moment of the monitored target and displays the walking movements of the monitored target.

[0176] Optionally, if the first monitoring information includes a three-dimensional model of the monitoring target, the monitoring information display module M130 is further used to:

[0177] In response to a first display operation instruction for the three-dimensional model, a display viewing angle state of the three-dimensional model is switched, wherein the display viewing angle state includes a dynamic viewing angle state and / or at least one fixed viewing angle state.

[0178] Optionally, if the first monitoring information includes a dynamic three-dimensional model of the monitoring target, the monitoring information display module M130 is further used to:

[0179] In response to a second display operation instruction for the three-dimensional model, the action state of the dynamic three-dimensional model is switched, and the action state includes a walking action state, a paused walking action state, and a standing state.

[0180] Optionally, the monitoring information display module M130 is further used for:

[0181] In response to the view detailed information instruction, at least part of the first monitoring information corresponding to the view detailed information instruction is magnified and displayed in a designated area of ​​a designated page.

[0182] It should be noted that the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. There may be other division methods in actual implementation, such as multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. The functional modules in the embodiments of the present invention may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may be stored in a computer-readable storage medium.

[0183] Since the principle of the monitoring target tracking and display device is the same as that of the monitoring target tracking and display method, the implementation of the monitoring target tracking and display device can refer to the implementation of the monitoring target tracking and display method described above, which will not be repeated here.

[0184] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, such as Fig.17As shown, it includes: a processor 110 and a memory 120 for storing executable instructions of the processor 110; wherein the processor 110 is configured to execute the instructions to implement the monitoring target tracking and display method.

[0185] In the specific implementation process, the device may have relatively large differences due to different configurations or performances, and may include one or more processors 110, memory 120, and computer-readable storage medium 130. The memory 120 and / or computer-readable storage medium 130 include one or more applications 131 or data 132. The memory 120 and / or computer-readable storage medium 130 may also include one or more operating systems 133, such as Windows, Mac OS, Linux, IOS, Android, Unix, FreeBSD, etc. Among them, the memory 120 and the computer-readable storage medium 130 may be short-term storage or persistent storage. The application 131 may include one or more modules ( Fig.17 ), each module may include a series of instruction operations. Further, the processor 110 may be configured to communicate with the computer-readable storage medium 130, and execute a series of instruction operations in the computer-readable storage medium 130 on the device. The device may also include one or more power supplies ( Fig.17 one or more network interfaces 140, wherein the network interface 140 includes a wired network interface 141 and / or a wireless network interface 142; and one or more input and output interfaces 143.

[0186] Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is used to implement the monitoring target tracking and display method.

[0187] The monitoring target tracking display method, device, equipment and computer storage medium provided by the embodiments of the present invention can avoid the failure of tracking due to the action of the monitoring target causing the monitoring image taken from a single angle to obtain effective information (for example, the monitoring target waves his arms while walking, causing his face to be blocked in the monitoring image taken from the side), by displaying the monitoring images taken from multiple angles by the multi-view surround camera group at the same shooting time in the form of a three-dimensional model, and facilitate the user to establish a three-dimensional feeling for the flat monitoring image, so as to better observe the monitoring target. In addition, by identifying the monitoring target to generate the first monitoring information and displaying it in the second area of ​​the first display interface, it is convenient for the user to obtain the information of the monitoring target with summary nature, so as to better observe the monitoring target.

[0188] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0189] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0192] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for monitoring target tracking and display, characterized in that: include: Obtain surveillance images taken by a multi-view surround camera group from N angles at the same shooting time, where N is a positive integer greater than 2; Establishing a straight N-prism stereo model, and displaying the straight N-prism stereo model on a first area of ​​a first display interface, wherein each side of the straight N-prism stereo model displays a monitoring image; Identifying a monitoring target image region where a monitoring target is located in the monitoring image, and a face image region where a face of the monitoring target is located; On the monitoring image of the side of the straight N-prism stereo model, display an animation marking the monitoring target image area and the face image area; Extracting the monitoring target image area and the face image area to obtain second monitoring information and determining the monitoring target corresponding to the second monitoring information, moving the second monitoring information to a designated position floating outside the straight N-prism model for display, and displaying a transition animation of the movement; The first monitoring information of the monitoring target is generated according to the recognition result, and the first monitoring information is displayed in the second area of ​​the first display interface.

2. The method according to claim 1, characterized in that The method further comprises at least one of the following steps: In response to the first model display adjustment instruction, the straight N-prism stereo model is rotated by a specified angle with the height line of the straight N-prism stereo model as a rotation axis; In response to the second model display adjustment instruction, the straight N-prism three-dimensional model is rotated by the angle indicated by the second model display adjustment instruction, with the axis indicated by the second model display adjustment instruction as the rotation axis.

3. The method according to claim 1, characterized in that Each side surface of the straight N-prism three-dimensional model displays a surveillance image obtained by cutting the captured surveillance image into a preset shape.

4. The method according to claim 1, characterized in that Generating first monitoring information of the monitoring target according to the recognition result, and displaying the first monitoring information on the second area of ​​the first display interface, including: Saving the second monitoring information; Determine whether the first monitoring information of the monitoring target exists in the second area; If the first monitoring information of the monitoring target does not exist in the second area, generating the first monitoring information of the monitoring target according to the second monitoring information, and displaying it at the reserved position in the second area, and generating a new reserved position; Displaying a transition animation of moving the second monitoring information of the monitoring target to the position of the first monitoring information of the monitoring target; The reserved position is the starting position or the ending position of the arrangement.

5. The method according to claim 4, characterized in that If the first monitoring information of the monitoring target exists in the second area, the method further includes: The first monitoring information of the monitoring target is updated according to the second monitoring information and then displayed.

6. The method according to claim 4 or 5, characterized in that The first monitoring information of the monitoring target includes at least one of the following: The monitoring target area and / or the face image area, wherein the monitoring target area and the face image area belong to the second monitoring information; Removing a background image or extracting a human body mask from the monitoring target area of ​​the second monitoring information to obtain a gait diagram of the monitoring target; A three-dimensional model of the monitoring target is obtained by modeling the monitoring target using the gait diagram of the monitoring target.

7. The method according to claim 6, characterized in that The three-dimensional model is a dynamic three-dimensional model that is obtained by modeling using a gait graph corresponding to at least one shooting moment of the monitored target and displays the walking movements of the monitored target.

8. The method according to claim 6, characterized in that If the first monitoring information includes a three-dimensional model of the monitoring target, the method further includes: In response to a first display operation instruction for the three-dimensional model, a display viewing angle state of the three-dimensional model is switched, wherein the display viewing angle state includes a dynamic viewing angle state and / or at least one fixed viewing angle state.

9. The method according to claim 7, characterized in that If the first monitoring information includes a dynamic three-dimensional model of the monitoring target, the method further includes: In response to a second display operation instruction for the three-dimensional model, the action state of the dynamic three-dimensional model is switched, and the action state includes a walking action state, a paused walking action state, and a standing state.

10. The method according to claim 4, characterized in that The method further comprises: In response to the view detailed information instruction, at least part of the first monitoring information corresponding to the view detailed information instruction is magnified and displayed in a designated area of ​​a designated page.

11. A device for monitoring target tracking and display, characterized in that: include: A monitoring image acquisition module is used to acquire monitoring images taken by a multi-view surround camera group from N angles at the same shooting time, where N is a positive integer greater than 2; A monitoring image display module, used for establishing a straight N-prism stereo model and displaying the straight N-prism stereo model on a first area of ​​a first display interface, wherein each side of the straight N-prism stereo model displays a monitoring image; A monitoring information display module, used to identify a monitoring target image area where a monitoring target is located in the monitoring image, and a face image area where the face of the monitoring target is located; and display an animation marking the monitoring target image area and the face image area on the monitoring image on the side of the straight N-prism stereo model; Extracting the monitoring target image area and the face image area to obtain second monitoring information and determining the monitoring target corresponding to the second monitoring information, moving the second monitoring information to a designated position floating outside the straight N-prism model for display, and displaying a transition animation of the movement; The first monitoring information of the monitoring target is generated according to the recognition result, and the first monitoring information is displayed in the second area of ​​the first display interface.

12. An electronic device, characterized in that: include: a processor and a memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the method for monitoring target tracking and display as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to implement the monitoring target tracking and display method according to any one of claims 1 to 10.

Images