System for viewing target based on linkage of AR panoramic camera and common camera

Abstract

The invention discloses a system for viewing a target based on linkage of an AR panorama camera and common cameras. The system comprises the AR panorama camera, the common cameras arranged at each monitoring node in a monitoring area, a server for receiving real-time pictures uploaded by the cameras and a screen for displaying the pictures of the cameras. By setting the polling mode and the monitoring mode, accurate viewing of the target is realized, and in the polling mode, when the preset target is detected, all common camera images with the preset target are displayed in a screen for a worker to view, so that the worker can conveniently process the preset target; in the monitoring mode, the label corresponding to any common camera in the panoramic picture can be manually clicked and checked, the user experience is improved, when the preset target is detected, the preset target appearing in the common camera picture displayed on the current screen is subjected to frame selection display, a worker can conveniently check the preset target, and the user experience is improved. And a worker can conveniently process the preset target.

Description

Technical Field

[0001] The present invention belongs to the field of target viewing, and in particular relates to a system for viewing targets based on an AR panoramic camera linked with an ordinary camera. Background Art

[0002] An AR panoramic camera is a camera that shoots at an ultra-wide angle. The pictures and videos taken are more three-dimensional and can capture image content with a larger field of view, while ordinary cameras can capture more accurate image details. Therefore, the linkage between AR panoramic cameras and ordinary cameras has been widely used in the security field. In the prior art, the image screen of an AR panoramic camera generally contains the shooting positions of one or more ordinary cameras. In order to achieve a quick jump from the AR panoramic screen to the ordinary camera, a jump entrance can be provided by setting a label in the AR panoramic screen, and the specific situation of the ordinary camera's location can be further viewed by clicking the label of the ordinary camera.

[0003] The above technology has the following defects: First, when the AR panoramic camera is linked with an ordinary camera for regional monitoring, it is generally combined with target detection technology, which can identify target tags and track motion. However, when viewing the target tag, it is necessary to manually switch to the ordinary camera at the corresponding position, and the monitoring screen cannot be automatically jumped, so there is a problem of inconvenience in use; second, the existing technology is based on preset jump plan rules, which makes the system unable to adapt to sudden security incidents or abnormal situations (i.e., a target intrudes into the monitoring area), resulting in reduced real-time and effectiveness of monitoring. Summary of the invention

[0004] The purpose of the present invention is to solve the problems raised in the background technology and to propose a system for viewing a target based on an AR panoramic camera linked with an ordinary camera.

[0005] To achieve the above object, the technical solution adopted by the present invention is: The present invention proposes a system for viewing a target based on an AR panoramic camera linked with an ordinary camera, comprising an AR panoramic camera, an ordinary camera arranged at each monitoring node in a monitoring area, a server receiving real-time images uploaded by each camera, and a screen for displaying images of each camera; The picture of the AR panoramic camera is a panoramic picture of the entire monitoring area. The server also sets each ordinary camera as a label to be displayed in the panoramic picture, where: The server continuously detects the images of all common cameras in real time; The server sets the first priority order for each ordinary camera according to the preset rules; The server displays the images of each camera on the screen in polling mode: When the server does not detect the preset target, the server switches the real-time images of each ordinary camera on the screen in sequence according to the first priority order; When the server detects the preset target, the server sets the order of the ordinary camera images where the preset target is located to the highest priority and displays them on the current screen, and displays all the ordinary camera images where the preset target is located evenly on the screen; Or the server displays the images of each camera on the screen according to the monitoring mode: Manually click on the labels corresponding to the ordinary cameras on the panoramic screen to switch the ordinary camera screen displayed on the current screen; When the server detects the preset target, the server will first highlight the label corresponding to the ordinary camera where the preset target is located in the panoramic picture, and then manually click on the first highlighted labels in sequence according to the first priority order to display the picture of the corresponding ordinary camera on the screen, and frame the preset target appearing in the ordinary camera picture currently displayed on the screen.

[0006] Preferably, the AR panoramic camera includes a surround lens and a gaze lens, the picture displayed in the surround lens is the panoramic picture and remains stationary, and the gaze lens changes with the change of the ordinary camera picture displayed on the current screen; The screen of the AR panoramic camera also includes a regional electronic map with additional coordinate information, and the regional electronic map screen is the screen displayed in the gaze lens. The coordinate information in the regional electronic map is obtained by establishing a plane rectangular coordinate system based on the position of the AR panoramic camera as the origin on the server side.

[0007] Preferably, in various modes, when the screen currently displays a picture of one of the ordinary cameras, the AR panoramic camera rotates the gaze lens by a corresponding angle toward the position coordinates of the ordinary camera; In polling mode, when the screen currently displays images equally shared by multiple ordinary cameras, the AR panoramic camera rotates the corresponding angles according to the priority order of the images of the multiple ordinary cameras so that the gaze lens faces the position coordinates of the corresponding ordinary camera.

[0008] Preferably, in the polling mode, the current ordinary camera image is displayed in full screen, the panoramic image and the regional electronic image are displayed in small windows of preset size on the screen, and the small windows are superimposed on the full screen.

[0009] Preferably, in monitoring mode, the current ordinary camera picture, panoramic picture and regional electronic picture are all displayed in a small window of a preset size on the screen.

[0010] Preferably, in various modes, the server performs a second prominent display on the panoramic picture of the label of the ordinary camera corresponding to the current picture displayed on the screen.

[0011] Preferably, the first priority order of each of the ordinary cameras is: sorting according to the size of the horizontal coordinate of the corresponding ordinary camera, and when the sizes of the horizontal coordinates are consistent, sorting according to the size of the vertical coordinate.

[0012] Preferably, in the polling mode, when the server detects that the preset target appears in multiple ordinary camera images at the same time, the multiple ordinary camera images are sequentially set as the highest priority according to the original first priority order and are evenly displayed on the screen.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention realizes accurate viewing of the target by setting two modes: polling mode and monitoring mode. In the polling mode, when a preset target is detected, all ordinary camera images with the preset target are displayed on the screen for the staff to view, so that the staff can handle the preset target conveniently; in the monitoring mode, the label corresponding to any ordinary camera in the panoramic picture can be manually clicked to view it, thereby increasing the user experience. When the preset target is detected, the preset target appearing in the ordinary camera picture currently displayed on the screen is framed and displayed, so that the staff can view the preset target conveniently and handle the preset target conveniently. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a module block diagram of a system for viewing a target based on an AR panoramic camera linked with a common camera according to the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0016] It should be noted that when a component is referred to as being "connected" to another component, it may be directly connected to the other component or there may be an intermediate component. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0017] like Figure 1 As shown, a system for viewing a target based on an AR panoramic camera linked with a common camera includes: The system for viewing target tags based on AR panoramic camera linkage with ordinary camera includes AR panoramic camera, ordinary camera deployed at each monitoring node in the monitoring area, a server receiving real-time images uploaded by each camera, and a screen for displaying the images of each camera; The picture of the AR panoramic camera is a panoramic picture of the entire monitoring area. The server also sets each ordinary camera as a label to display in the panoramic picture (each ordinary camera can be numbered, and the server will display the corresponding ordinary camera number on the regional electronic map), where: The server continuously detects the images of all common cameras in real time; The server sets the first priority order for each ordinary camera according to the preset rules; The server displays the images of each camera on the screen in polling mode: When the server does not detect the preset target, the server switches the real-time images of each ordinary camera on the screen in sequence according to the first priority order; When the server detects the preset target, the order of the ordinary camera images where the preset target is located is set to the highest priority and displayed on the current screen, and all the ordinary camera images where the preset target is located are evenly displayed on the screen; Or the server displays the images of each camera on the screen according to the monitoring mode: Manually click on the labels corresponding to the ordinary cameras on the panoramic screen to switch the ordinary camera screen displayed on the current screen; When the server detects the preset target, the server will first highlight the label corresponding to the ordinary camera where the preset target is located in the panoramic picture, and then manually click on the first highlighted labels in sequence according to the first priority order to display the picture of the corresponding ordinary camera on the screen, and frame the preset target appearing in the ordinary camera picture currently displayed on the screen.

[0018] It should be noted that the entire monitoring area is divided into a preset number of small areas (the specific number is set according to the actual needs of the entire monitoring area), a position is selected in each small area as a monitoring node (such as selecting the center point of each small area as the monitoring node), and an ordinary camera is deployed on each monitoring node. The entire monitoring area is monitored by the AR panoramic camera (the position of the AR panoramic camera can be set according to actual needs, such as setting it at the center of the entire monitoring area). The server uses the AR panoramic camera as the origin to establish a plane rectangular coordinate system for the entire monitoring area; the server sets a first priority order for all ordinary cameras, specifically: the first priority order of each ordinary camera is: sort according to the size of the horizontal coordinate of the corresponding ordinary camera, and when the horizontal coordinates are the same in size, sort them according to the size of the vertical coordinate (such as first sorting the horizontal coordinates of each ordinary camera by size, the larger the horizontal coordinate, the higher the priority, and when the horizontal coordinates are the same in size, sorting the vertical coordinates by size, the larger the vertical coordinate, the higher the priority; but the specific priority setting rules are not limited, and other rules can also be used).

[0019] The AR panoramic camera includes a surround lens and a gaze lens. The image displayed in the surround lens is the panoramic image and remains stationary, while the gaze lens changes with the ordinary camera image displayed on the current screen. The picture of the AR panoramic camera also includes a regional electronic map with additional coordinate information, and the picture of the regional electronic map is the picture displayed in the gaze lens. The coordinate information in the regional electronic map is obtained by establishing a plane rectangular coordinate system with the position of the AR panoramic camera as the origin by the server (the server displays the picture of the gaze lens as a regional electronic map, and displays the coordinate information in the regional electronic map).

[0020] Among them, in various modes, when the screen currently displays the image of one of the ordinary cameras, the AR panoramic camera will rotate the gaze lens to a corresponding angle toward the position coordinates of the ordinary camera; that is, the lens angle of the AR panoramic camera changes with the coordinate transformation of the ordinary camera displayed on the screen. When the screen displays the image of one of the ordinary cameras, the server calculates the angle value at which the AR panoramic camera turns the gaze lens toward the ordinary camera based on the coordinate positions of the ordinary camera and the AR panoramic camera, and the current angle of the gaze lens of the AR panoramic camera, and then controls the gaze lens of the AR panoramic camera to rotate to a corresponding angle (the calculation process belongs to the existing technology and will not be described in detail), so that the image of the regional electronic map is a real-time real scene image centered on the ordinary camera.

[0021] During the entire monitoring process, the server performs real-time detection on the images of all ordinary cameras (and the detection is continuous throughout the monitoring process). The specific real-time detection process can use a target detection model (such as a convolutional neural network model), which belongs to the existing technology and will not be described in detail. The preset target can be a suspicious person or object, etc.

[0022] The target viewing methods based on the AR panoramic camera linked with the ordinary camera include polling mode and monitoring mode (the monitoring mode can be set to polling mode or monitoring mode according to actual needs): Among them, in the polling mode (automatic mode), the current ordinary camera image is displayed in full screen, and the panoramic image and regional electronic map are displayed in small windows of preset size on the screen, and the small windows are superimposed on the full screen (the size of the two small windows can be set according to actual needs without specific restrictions, the position of each small window can be set at the corner of the screen, and the transparency of each small window can be adjusted without affecting the full screen display); In the case that the server does not detect the target, the server sets the real-time images of each ordinary camera to switch the images on the screen in sequence according to the first priority order (full-screen display, the specific switching interval can be set independently according to actual needs, such as switching every 5 seconds, and the real-time image at the current moment is displayed); when the server detects the preset target, the server sets the order of the ordinary camera images where the preset target is located to the highest priority and displays them on the current screen, and displays all the ordinary camera images where the preset target is located evenly on the screen (during the entire monitoring process, the ordinary camera images where the preset target is located detected in sequence are accumulated on the screen, and all the accumulated ordinary camera images are evenly displayed on the screen, and are evenly displayed within the range where the ordinary camera images are displayed in full screen), so that the staff can watch the preset target. At this time, the staff needs to manually close the evenly displayed images, and then the content in the full screen returns to the ordinary cameras to switch the images on the screen in sequence according to the first priority order; When the server detects that the preset target appears in multiple ordinary camera images at the same time, the multiple ordinary camera images are set to the highest priority in turn according to the original first priority order and are evenly displayed on the screen; In polling mode, when the screen currently displays images equally shared by multiple ordinary cameras, the AR panoramic camera rotates the corresponding angles according to the priority order of the images of the multiple ordinary cameras so that the gaze lens faces the position coordinates of the corresponding ordinary camera.

[0023] Among them, in monitoring mode, the current ordinary camera image, panoramic image and regional electronic map are all displayed in small windows of preset sizes on the screen (the three windows are of equal size and evenly displayed on the same screen. You can click one of the windows to display it in full screen, which is convenient for staff to view the enlarged image, and you can click again to return to the small window state); When the server does not detect the target, the staff can manually click on the labels corresponding to the ordinary cameras on the panoramic screen to switch the ordinary camera images currently displayed on the screen; when the server detects the preset target, the server will first highlight the label corresponding to the ordinary camera where the preset target is located in the panoramic screen (such as using different colors for highlighting to facilitate the staff to click and view), and then manually click on the first highlighted labels in the first priority order to display the corresponding ordinary camera images on the screen, and frame the preset target appearing in the ordinary camera image currently displayed on the screen.

[0024] In various modes, the server will display the label of the ordinary camera corresponding to the current picture displayed on the screen in a second prominent manner in the panoramic picture (such as flashing display, so that it is easy to know which ordinary camera's picture is currently being viewed).

[0025] In this system, AR panoramic cameras are used to capture panoramic monitoring images and regional electronic map images, and ordinary cameras are used to capture detailed monitoring images at monitoring nodes, which are then transmitted to the background screen through data transmission equipment for real-time monitoring.

[0026] The target detection model automatically collects sample data and optimizes the target detection model through online learning. The sample information includes positive sample information and negative sample information. The positive sample information specifically includes the target object image and annotation information, and the negative sample information specifically includes images that do not contain the target object and images that are similar in appearance to the target object.

[0027] The present invention realizes accurate viewing of the target by setting two modes: polling mode and monitoring mode. In the polling mode, when a preset target is detected, all ordinary camera images with the preset target are displayed on the screen for the staff to view, so that the staff can handle the preset target conveniently; in the monitoring mode, the label corresponding to any ordinary camera in the panoramic picture can be manually clicked to view it, thereby increasing the user experience. When the preset target is detected, the preset target appearing in the ordinary camera picture currently displayed on the screen is framed and displayed, so that the staff can view the preset target conveniently and handle the preset target conveniently.

[0028] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0029] The above-described embodiments only express the more specific and detailed embodiments described in this application, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this application, which all belong to the protection scope of this application. Therefore, the protection scope of the patent application shall be based on the attached claims.

Claims

1. A system for viewing a target based on an AR panoramic camera linked with an ordinary camera, characterized in that: The system for viewing target tags based on AR panoramic camera linkage with ordinary camera includes an AR panoramic camera, ordinary cameras deployed at each monitoring node in the monitoring area, a server receiving real-time images uploaded by each camera, and a screen for displaying the images of each camera; The picture of the AR panoramic camera is a panoramic picture of the entire monitoring area. The server also sets each ordinary camera as a label to be displayed in the panoramic picture, where: The server continuously detects the images of all common cameras in real time; The server sets the first priority order for each ordinary camera according to the preset rules; The server displays the images of each camera on the screen in polling mode: When the server does not detect the preset target, the server switches the real-time images of each ordinary camera on the screen in sequence according to the first priority order; When the server detects the preset target, the server sets the order of the ordinary camera images where the preset target is located to the highest priority and displays them on the current screen, and displays all the ordinary camera images where the preset target is located evenly on the screen; Or the server displays the images of each camera on the screen according to the monitoring mode: Manually click on the labels corresponding to the ordinary cameras on the panoramic screen to switch the ordinary camera screen displayed on the current screen; When the server detects the preset target, the server will first highlight the label corresponding to the ordinary camera where the preset target is located in the panoramic picture, and then manually click on the first highlighted labels in sequence according to the first priority order to display the picture of the corresponding ordinary camera on the screen, and frame the preset target appearing in the ordinary camera picture currently displayed on the screen.

2. The system for viewing a target based on an AR panoramic camera linked with a common camera as claimed in claim 1, characterized in that: The AR panoramic camera includes a surround lens and a gaze lens. The picture displayed in the surround lens is the panoramic picture and remains stationary, and the gaze lens changes with the change of the ordinary camera picture displayed on the current screen; The screen of the AR panoramic camera also includes a regional electronic map with additional coordinate information, and the regional electronic map screen is the screen displayed in the gaze lens. The coordinate information in the regional electronic map is obtained by establishing a plane rectangular coordinate system based on the position of the AR panoramic camera as the origin on the server side.

3. The system for viewing a target based on an AR panoramic camera linked with a common camera as claimed in claim 2, characterized in that: In various modes, when the screen currently displays the image of one of the ordinary cameras, the AR panoramic camera will rotate the gaze lens to the corresponding angle toward the position coordinates of the ordinary camera; In polling mode, when the screen currently displays images equally shared by multiple ordinary cameras, the AR panoramic camera rotates the corresponding angles according to the priority order of the images of the multiple ordinary cameras so that the gaze lens faces the position coordinates of the corresponding ordinary camera.

4. The system for viewing a target based on an AR panoramic camera linked with a common camera as claimed in claim 2, characterized in that: In polling mode, the current ordinary camera image is displayed in full screen, and the panoramic image and regional electronic map are displayed in small windows of preset size on the screen, and the small windows are superimposed on the full screen.

5. The system for viewing a target based on an AR panoramic camera linked with a common camera as claimed in claim 2, characterized in that: In monitoring mode, the current ordinary camera image, panoramic image and regional electronic map are displayed on the screen in a small window of preset size.

6. The system for viewing a target based on an AR panoramic camera linked with a common camera as claimed in claim 1, characterized in that: In various modes, the server displays the label of the ordinary camera corresponding to the current picture displayed on the screen in a second prominent manner in the panoramic picture.

7. The system for viewing a target based on an AR panoramic camera linked with a common camera as claimed in claim 1, characterized in that: The first priority order of each of the ordinary cameras is: sorting according to the size of the horizontal coordinates of the corresponding ordinary cameras, and when the sizes of the horizontal coordinates are the same, sorting according to the size of the vertical coordinates.

8. The system for viewing a target based on an AR panoramic camera linked with a common camera as claimed in claim 1, characterized in that: In the polling mode, when the server detects that the preset target appears in multiple ordinary camera images at the same time, the multiple ordinary camera images are set to the highest priority in turn according to the original first priority order and are evenly displayed on the screen.

Images