An event positioning method, apparatus and electronic device

Abstract

Embodiments of the present application provide an event positioning method and device and electronic equipment, and relate to the technical field of visual positioning. The method comprises: detecting a first side-view visible light image of a to-be-detected scene area for a preset type of event to obtain an event detection result; in the case where the preset type of event exists, obtaining a to-be-matched side-view visible light image and a DEM side view of the to-be-detected scene area; obtaining a first image position in the first side-view visible light image where the preset type of event occurs, determining a second image position corresponding to the first image position in the to-be-matched side-view visible light image; matching the same scene points in the to-be-matched side-view visible light image and the DEM side view to obtain a scene point matching result; performing dense mapping on the to-be-matched side-view visible light image and the DEM side view to obtain a dense mapping relationship; and determining three-dimensional position information corresponding to the second image position. The embodiments of the present application improve the accuracy of positioning information in a large scene.

Description

Technical Field

[0001] This application relates to the field of visual positioning technology, and in particular to an event positioning method, apparatus and electronic device. Background Technology

[0002] Large, sparsely populated areas such as forests and valleys are difficult to locate. In the event of an emergency, achieving long-distance, high-precision monitoring and location of the incident to promptly resolve the crisis has always been a pressing challenge. For example, in the event of a fire or other emergency, pinpointing the location of the fire has always been a difficult aspect of fire suppression.

[0003] Most of the positioning methods proposed in related technologies have problems such as difficulty in implementation and low positioning accuracy. For example, the scheme of using high-point cameras installed on watchtowers to monitor the fire point of a large-scale fire using video still cannot accurately determine the location of the fire point due to geographical reasons. Summary of the Invention

[0004] The purpose of this application is to provide an event localization method, apparatus, and electronic device to improve the accuracy of localization information in large-scale scenarios. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide an event location method, including:

[0006] The first side-view visible light image of the scene area to be detected is used to detect events of a preset type, and the event detection results are obtained.

[0007] If the event detection result indicates that a preset type of event exists in the first side-view visible light image, the side-view visible light image to be matched for the scene area to be detected, and the DEM side view are obtained, wherein the DEM side view includes the three-dimensional position information of each point in the scene area to be detected;

[0008] Obtain the first image position in the first side-view visible light image where a preset type of event occurs, and determine the second image position corresponding to the first image position in the side-view visible light image to be matched;

[0009] Match the same scene points in the side-view visible light image to be matched and the side view of the DEM to obtain scene point matching results;

[0010] Based on the scene point matching results, dense mapping is performed on the side view visible light image to be matched and the DEM side view to obtain a dense mapping relationship.

[0011] Based on the three-dimensional position information of each point in the DEM side view and the dense mapping relationship, the three-dimensional position information corresponding to the second image position is determined.

[0012] In one embodiment of this application, when the event detection result indicates that a preset type of event exists in the first side-view visible light image, acquiring the side-view visible light image to be matched for the scene region to be detected, and the DEM side view, includes:

[0013] If the event detection result indicates that a preset type of event exists in the first side-view visible light image, a side-view panoramic visible light image of the scene area to be detected, and a DEM side view are obtained.

[0014] In one embodiment of this application, when the event detection result indicates that a preset type of event exists in the first side-view visible light image, acquiring a side-view panoramic visible light image of the scene area to be detected, and a DEM side view, includes:

[0015] When the event detection result indicates that a preset type of event exists in the first side-view visible light image, second side-view visible light images of the scene area to be detected are acquired from multiple different viewpoints, wherein adjacent viewpoints have overlapping fields of view;

[0016] Based on the acquisition angle of each of the second side-view visible light images, the second side-view visible light images are stitched together in a panoramic view to obtain a side-view panoramic visible light image.

[0017] Obtain the DEM image of the scene area to be detected, wherein the DEM image includes the elevation information and latitude and longitude information of each point in the scene area to be detected;

[0018] A DEM side view is generated based on the elevation and latitude / longitude information of each point in the DEM image, and the elevation and latitude / longitude information of the visible light camera that acquired the second side view visible light image.

[0019] In one embodiment of this application, obtaining the first image position in the first side-view visible light image where a preset type event occurs, and determining the second image position corresponding to the first image position in the side-view panoramic visible light image, includes:

[0020] Acquire at least one auxiliary image of the first side-view visible light image, first view information of the first side-view visible light image, and second view information of the auxiliary image; wherein, the field of view of the auxiliary image includes the field of view of the scene area corresponding to the position of the first image, and the magnification of the auxiliary image is less than the magnification of the first side-view visible light image.

[0021] Based on the first perspective information and the second perspective information, determine the third image position corresponding to the first image position in the auxiliary image;

[0022] Based on the second perspective information and the third image position, the second image position corresponding to the first image position in the side-view panoramic visible light image is determined.

[0023] In one embodiment of this application, the first perspective information is first PTZ information;

[0024] The step of acquiring at least one auxiliary image of the first side-view visible light image, first viewpoint information of the first side-view visible light image, and second viewpoint information of the auxiliary image includes:

[0025] Obtain N auxiliary images of the first side-view visible light image, the first view angle information of the first side-view visible light image, and the PTZ information of the auxiliary images, wherein N is an integer greater than 1, the P and T information of the first side-view visible light image is the same as that of the first N-1 auxiliary images, the magnification of the N-1th auxiliary image is the same as that of the Nth auxiliary image, the magnification of the 1st auxiliary image is greater than that of the first side-view visible light image, the magnification of the (i+1)th auxiliary image is greater than that of the 1st auxiliary image, i∈[1, N-1], and i is an integer;

[0026] Determining the third image position corresponding to the first image position in the auxiliary image based on the first viewpoint information and the second viewpoint information includes:

[0027] Based on the first PTZ information and the PTZ information of the first auxiliary image, determine the mapping position of the scene region corresponding to the position of the first image in the first auxiliary image;

[0028] Based on the PTZ information of the i-th and i+1-th auxiliary images, the mapping position of the scene region corresponding to the first image position in the i+1-th auxiliary image is determined, wherein the third image position is the mapping position of the scene region corresponding to the first image position in the N-th auxiliary image.

[0029] In one embodiment of this application, the second side-view visible light image includes the first side-view visible light image;

[0030] The step of obtaining the first image position in the first side-view visible light image where a preset type event occurs, and determining the second image position corresponding to the first image position in the side-view panoramic visible light image, includes:

[0031] Obtain the first image location in the first side-view visible light image where a preset type of event occurs, and the coordinate transformation relationship between the image coordinate system of the first side-view visible light image and the image coordinate system of the side-view panoramic visible light image;

[0032] According to the coordinate transformation relationship, the first image position is converted into the second image position in the side-view panoramic visible light image.

[0033] In one embodiment of this application, the step of performing dense mapping on the side-view visible light image to be matched and the DEM side view based on the scene point matching result to obtain a dense mapping relationship includes:

[0034] The side-view visible light image to be matched is divided into multiple visible light regions according to the scene points, and the DEM side view is divided into multiple DEM side view regions.

[0035] Based on the scene point matching results, the mapping transformation matrix between the visible light region and the side view region of the DEM with matching relationship is calculated to obtain the dense mapping relationship.

[0036] Secondly, embodiments of this application provide an event location device, comprising:

[0037] The event detection module is used to detect preset types of events in the first side view visible light image of the scene area to be detected, and obtain the event detection results;

[0038] The image acquisition module is used to acquire the matching side-view visible light image of the scene area to be detected and the DEM side view when the event detection result indicates that there is a preset type of event in the first side-view visible light image. The DEM side view includes the three-dimensional position information of each point in the scene area to be detected.

[0039] The image position determination module is used to obtain the first image position in the first side-view visible light image where a preset type of event occurs, and to determine the second image position corresponding to the first image position in the side-view visible light image to be matched.

[0040] The scene point matching module is used to match the same scene points in the side-view visible light image to be matched and the side view of the DEM to obtain the scene point matching result;

[0041] The dense mapping module is used to perform dense mapping on the side-view visible light image to be matched and the DEM side view based on the scene point matching result, so as to obtain a dense mapping relationship;

[0042] The location information determination module is used to determine the three-dimensional location information corresponding to the second image location based on the three-dimensional location information of each point in the DEM side view and the dense mapping relationship.

[0043] In one embodiment of this application, the image acquisition module includes:

[0044] The image acquisition submodule is used to acquire a side-view panoramic visible light image of the scene area to be detected, and a DEM side view, when the event detection result indicates that a preset type of event exists in the first side-view visible light image.

[0045] In one embodiment of this application, the image acquisition submodule is specifically used for:

[0046] When the event detection result indicates that a preset type of event exists in the first side-view visible light image, second side-view visible light images of the scene area to be detected are acquired from multiple different viewpoints, wherein adjacent viewpoints have overlapping fields of view;

[0047] Based on the acquisition angle of each of the second side-view visible light images, the second side-view visible light images are stitched together in a panoramic view to obtain a side-view panoramic visible light image.

[0048] Obtain the DEM image of the scene area to be detected, wherein the DEM image includes the elevation information and latitude and longitude information of each point in the scene area to be detected;

[0049] A DEM side view is generated based on the elevation and latitude / longitude information of each point in the DEM image, and the elevation and latitude / longitude information of the visible light camera that acquired the second side view visible light image.

[0050] In one embodiment of this application, the image position determination module includes:

[0051] An information acquisition submodule is used to acquire at least one auxiliary image of the first side-view visible light image, first view information of the first side-view visible light image, and second view information of the auxiliary image; wherein, the field of view of the auxiliary image includes the field of view of the scene area corresponding to the position of the first image, and the magnification of the auxiliary image is less than the magnification of the first side-view visible light image.

[0052] The third image position determination submodule is used to determine the third image position corresponding to the first image position in the auxiliary image based on the first view information and the second view information.

[0053] The second image position determination submodule is used to determine the second image position corresponding to the first image position in the side-view panoramic visible light image based on the second view information and the third image position.

[0054] In one embodiment of this application, the first perspective information is first PTZ information;

[0055] The information acquisition submodule is specifically used for:

[0056] Obtain N auxiliary images of the first side-view visible light image, the first view angle information of the first side-view visible light image, and the PTZ information of the auxiliary images, wherein N is an integer greater than 1, the P and T information of the first side-view visible light image is the same as that of the first N-1 auxiliary images, the magnification of the N-1th auxiliary image is the same as that of the Nth auxiliary image, the magnification of the 1st auxiliary image is greater than that of the first side-view visible light image, the magnification of the (i+1)th auxiliary image is greater than that of the 1st auxiliary image, i∈[1, N-1], and i is an integer;

[0057] The third image position determination submodule is specifically used for:

[0058] Based on the first PTZ information and the PTZ information of the first auxiliary image, determine the mapping position of the scene region corresponding to the position of the first image in the first auxiliary image;

[0059] Based on the PTZ information of the i-th and i+1-th auxiliary images, the mapping position of the scene region corresponding to the first image position in the i+1-th auxiliary image is determined, wherein the third image position is the mapping position of the scene region corresponding to the first image position in the N-th auxiliary image.

[0060] In one embodiment of this application, the second side-view visible light image includes the first side-view visible light image;

[0061] The image position determination module is specifically used for:

[0062] Obtain the location of the first image in the first side-view visible light image where a preset type of event occurs, and the coordinate transformation relationship between the image coordinate system of the first side-view visible light image and the image coordinate system of the side-view panoramic visible light image;

[0063] According to the coordinate transformation relationship, the first image position is converted into the second image position in the side-view panoramic visible light image.

[0064] In one embodiment of this application, the dense mapping module is specifically used for:

[0065] The side-view visible light image to be matched is divided into multiple visible light regions according to the scene points, and the DEM side view is divided into multiple DEM side view regions.

[0066] Based on the scene point matching results, the mapping transformation matrix between the visible light region and the side view region of the DEM with matching relationship is calculated to obtain the dense mapping relationship.

[0067] Thirdly, embodiments of this application also provide an electronic device, including:

[0068] Memory, used to store computer programs;

[0069] When a processor executes a program stored in memory, it implements any of the event location methods described above.

[0070] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the event location methods described above.

[0071] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the event location methods described above.

[0072] Beneficial effects of the embodiments in this application:

[0073] The event localization method provided in this application, by densely mapping the scene points between the side-view visible light image to be matched and the DEM side view, obtains the location information of each scene point with high positioning accuracy. Even in large scene areas at long distances, if the number of matching points under dense mapping meets a certain requirement, high positioning accuracy can still be achieved. Based on this, the positioning information of the occurrence of a preset type of event is obtained, thereby improving the accuracy of positioning information in large scenes. In addition, when the event detection result indicates that a preset type of event exists in the first side-view visible light image, the first image position of the occurrence of the preset type of event in the first side-view visible light image is used to determine the second image position corresponding to the first image position in the side-view visible light image to be matched, and then the corresponding three-dimensional position information is determined based on the second image position. Since this is an online event localization process, it can avoid unexpected situations such as camera repositioning errors, increased positioning errors due to changes in camera extrinsic parameters, and calibration data failure that may be caused by offline calibration in advance, further improving the accuracy of positioning information in large scenes and enhancing the long-term positioning accuracy and reliability of the solution.

[0074] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0076] Figure 1-1 A flowchart illustrating an event localization method provided in an embodiment of this application;

[0077] Figure 1-2 An example diagram of a DEM image provided in an embodiment of this application;

[0078] Figure 1-3 An example diagram of scene point matching provided in an embodiment of this application;

[0079] Figure 2-1 This is one possible implementation of step S102 provided in the embodiments of this application;

[0080] Figure 2-2 An example diagram of a single-view visible light image provided in an embodiment of this application;

[0081] Figure 2-3 An example diagram of panoramic visible light image stitching provided in this application embodiment;

[0082] Figure 2-4 An example diagram of a DEM side view provided for an embodiment of this application;

[0083] Figure 2-5 An example diagram of a side-view panoramic visible light image provided in an embodiment of this application;

[0084] Figure 3 This is one possible implementation of step S103 provided in the embodiments of this application;

[0085] Figure 4 Another possible implementation of step S103 provided in the embodiments of this application;

[0086] Figure 5-1 This is one possible implementation of step S105 provided in the embodiments of this application;

[0087] Figure 5-2 An example diagram of a region division provided in an embodiment of this application;

[0088] Figure 5-3 A comparative example of a side-view panoramic visible light image and a densely mapped DEM side view is provided for embodiments of this application;

[0089] Figure 5-4 A flowchart illustrating an event localization method provided in an embodiment of this application;

[0090] Figure 6 This is a schematic diagram of the structure of an event location device provided in an embodiment of this application;

[0091] Figure 7This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0092] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0093] Since it is difficult to achieve accurate positioning in large-scale scenarios in related technologies, this application provides an event positioning method, device, and electronic device to solve this problem.

[0094] The following detailed description is provided through specific embodiments.

[0095] The event location method provided in this application can be applied to any electronic device with computing capabilities, such as servers, cameras, hard disk recorders, personal computers, etc.

[0096] Firstly, such as Figure 1-1 As shown in the figure, this application provides a flowchart of an event location method, including:

[0097] Step S101: Detect preset type events in the first side-view visible light image of the scene area to be detected, and obtain the event detection results.

[0098] The area to be monitored can be any area that needs to be monitored, such as forests, valleys, wetlands, nature reserves, etc. The preset event types are events that need to be located within the area to be monitored, such as natural disaster events like fires, landslides, and flash floods. Preset event types can also include human activity events, such as the presence of humans in uninhabited areas or wildlife reserves. The first side-view visible light image is an image captured by the image acquisition device used to monitor the area to be monitored, such as real-time video screenshots from surveillance equipment or snapshots taken at fixed intervals by surveillance equipment.

[0099] The first-side view of the visible light image of the scene area to be detected is used to detect events of a preset type. The event detection results reflect whether the preset type of event that needs to be located exists in the scene area. The detection of preset type events in the visible light image can be achieved by deep learning algorithms, and this application does not specifically limit the deep learning algorithm used.

[0100] Step S102: If the event detection result indicates that there is a preset type of event in the first side-view visible light image, obtain the side-view visible light image to be matched for the scene area to be detected, and the DEM side view.

[0101] The DEM side view includes the three-dimensional position information of each point in the scene area to be detected.

[0102] The side-view visible light image to be matched is an image that can clearly show the location of the preset type of event. Specifically, when there is a single side-view visible light image of the scene area to be detected that can clearly show the location of the preset type of event, the single side-view visible light image is used as the side-view visible light image to be matched.

[0103] In one embodiment of the present invention, when there is no single side-view visible light image of the scene area to be detected that can clearly show the location of the event of the preset type, a side-view panoramic visible light image of the scene area to be detected is obtained, and the side-view panoramic visible light image is used as the side-view visible light image to be matched.

[0104] A panoramic visible light image is a 360-degree optical image of the scene area to be inspected in the vertical direction. It can display the 360-degree view of the scene area in two-dimensional form through wide-angle representation. A side-view panoramic visible light image is a side view image of the panoramic mode of the scene area to be inspected. Specifically, the side-view panoramic visible light image of the scene area to be inspected can be acquired by any optical imaging device with panoramic image acquisition capabilities, or it can be obtained by stitching together multiple non-panoramic visible light images; there is no limitation here.

[0105] A DEM (Digital Elevation Model) can reflect local topographic features at a certain resolution and can be used to extract a large amount of surface morphology information. It is an important source of raw data for studying and analyzing topography, watersheds, and feature identification, for example... Figure 1-2 As shown, a DEM (Digital Image Model) is a bird's-eye view image. The image coordinates within it can be converted into GPS (Global Positioning System) coordinates using the transformation relationships stored within the DEM itself. The corresponding stored value is the elevation (height) of that point, thus enabling precise positioning of the scene within the DEM image in the real 3D world. The DEM image includes the 3D position information of each point. Furthermore, the DEM image serves as fundamental data for a large scene area and can be obtained from any relevant DEM image acquisition method.

[0106] The DEM side view of the scene area to be detected is a side view image of the scene area in the form of DEM. The scene area corresponding to the DEM side view is the same as that of the side-view panoramic visible light image. Both are 360-degree views of the scene area to be detected. They are panoramic views of the scene area to be detected in the vertical direction. The DEM side view also includes the location information of each point in the scene area to be detected, such as the GPS coordinates and elevation of each point.

[0107] Step S103: Obtain the first image position in the first side-view visible light image where a preset type of event occurs, and determine the second image position corresponding to the first image position in the side-view visible light image to be matched;

[0108] The image position in the first side-view visible light image where a preset type of event occurs is obtained, i.e., the first image position. The first side-view visible light image is compared with the side-view panoramic visible light image to determine the second image position in the side-view panoramic visible light image corresponding to the first image position.

[0109] Both the first side-view visible light image and the side-view panoramic visible light image are images of the scene area to be detected. Therefore, the coordinate transformation relationship between the first side-view visible light image and the side-view panoramic visible light image can be obtained based on their perspectives.

[0110] In one example, the second image position can be obtained by mapping the first image position after feature point matching between the first side-view visible light image and the side-view panoramic visible light image.

[0111] Step S104: Match the same scene points in the side-view visible light image to be matched and the side view of the DEM to obtain scene point matching results.

[0112] Matching identical scene points in the side-view panoramic visible light image and the DEM side view yields a one-to-one matching relationship between scene points in both images. Specifically, preliminary positioning can be achieved in the DEM side view based on obvious features of the scene region in the side-view panoramic visible light image, such as ridgeline trends. Based on the continuity of scene regions in both images, the matching region is first determined, followed by precise matching. Precise matching can be implemented using any common machine vision or deep learning-based methods and feature point matching algorithms, such as SIFT (Scale Invariant Feature Transform) and ORB (Oriented Fast and Rotated Brief). Manual recalibration can also be performed after algorithm calibration. Figure 1-3As shown. The scene point matching result obtained at this time is a sparse mapping between the side-view panoramic visible light image and the DEM side view.

[0113] Step S105: Based on the scene point matching result, perform dense mapping on the side view visible light image to be matched and the DEM side view to obtain a dense mapping relationship.

[0114] After obtaining the scene point matching results between the side-view panoramic visible light image and the DEM side view, a dense mapping is performed on the side-view panoramic visible light image and the DEM side view. This can be achieved using any common dense mapping algorithm, such as interpolation mapping algorithms, Dense-SIFT algorithms, and kNN (k-nearest neighbor classification) algorithms. This results in the dense mapping relationship between each scene point between the side-view panoramic visible light image and the DEM side view.

[0115] Step S106: Determine the three-dimensional position information corresponding to the second image position based on the three-dimensional position information of each point in the DEM side view and the dense mapping relationship.

[0116] Based on the dense mapping relationship between scene points in the side-view panoramic visible light image and the DEM side view, each point in the DEM side view is mapped to the position of the second image in the side-view panoramic visible light image. Given the position information of each point in the DEM side view, the position information of the second image in the DEM side view can be determined based on the dense mapping relationship. Then, based on the DEM side view, the three-dimensional position information corresponding to the position of the second image can be determined, thereby obtaining the three-dimensional coordinates of the location where the preset type of event occurs in the real world.

[0117] The event localization method provided in this application, by densely mapping the scene points between the side-view visible light image to be matched and the DEM side view, obtains the location information of each scene point with high positioning accuracy. Even in large scene areas at long distances, if the number of matching points under dense mapping meets a certain requirement, high positioning accuracy can still be achieved. Based on this, the positioning information of the occurrence of a preset type of event is obtained, thereby improving the accuracy of positioning information in large scenes. In addition, when the event detection result indicates that a preset type of event exists in the first side-view visible light image, the first image position of the occurrence of the preset type of event in the first side-view visible light image is used to determine the second image position corresponding to the first image position in the side-view visible light image to be matched, and then the corresponding three-dimensional position information is determined based on the second image position. Since this is an online event localization process, it can avoid unexpected situations such as camera repositioning errors, increased positioning errors due to changes in camera extrinsic parameters, and calibration data failure that may be caused by offline calibration in advance, further improving the accuracy of positioning information in large scenes and enhancing the long-term positioning accuracy and reliability of the solution.

[0118] In one embodiment of this application, such as Figure 2-1 As shown, step S102 above, when the event detection result indicates that a preset type of event exists in the first side-view visible light image, acquires a side-view panoramic visible light image of the scene area to be detected, and a DEM side view, including:

[0119] Step S201: If the event detection result indicates that there is a preset type of event in the first side-view visible light image, acquire second side-view visible light images of the scene area to be detected from multiple different viewpoints.

[0120] In this case, adjacent viewpoints have overlapping fields of view.

[0121] Second-side visible light images of the scene area to be detected were acquired from multiple different perspectives, such as... Figure 2-2 As shown, the second side-view visible light image has a certain field of view overlap between adjacent viewpoints.

[0122] Step S202: Based on the acquisition angle of each of the second side-view visible light images, perform panoramic stitching on each of the second side-view visible light images to obtain a side-view panoramic visible light image.

[0123] Based on the acquisition perspective of each second side-view visible light image, the second side-view visible light images with overlapping fields of view are treated as adjacent images and stitched together in a panoramic view to obtain a side-view panoramic visible light image, for example... Figure 2-3 As shown, the area within the box in the figure is Figure 2-2 Mid-scene area.

[0124] Step S203: Obtain the DEM image of the scene area to be detected;

[0125] The DEM image includes elevation and latitude / longitude information of each point in the scene area to be detected.

[0126] Step S204: Generate a DEM side view based on the elevation and latitude / longitude information of each point in the DEM image and the elevation and latitude / longitude information of the visible light camera that acquired the second side-view visible light image.

[0127] The side view of the DEM of the scene region to be detected is obtained by converting the DEM image of the scene region to a side view from the camera's perspective, such as... Figure 2-4 As shown, Figure 1-2 The DEM side view of the mid-field attractions has a horizontal field of view covering 360 degrees. The corresponding coordinate values ​​can store GPS coordinates, elevation information, and converted XYZ coordinates in a Cartesian coordinate system with the camera as the origin. Figure 2-5 and Figure 2-3 The images are a side view (360-degree field of view) of the DEM and a side-view panoramic visible light image of the same scene area. The scene area in the side-view panoramic visible light image corresponds to the area within the frame of the DEM side view.

[0128] Specifically, the DEM image includes elevation and latitude / longitude information for each point in the scene area to be detected. The visible light camera that acquired the side-view visible light images of each sample also has elevation and latitude / longitude information. Based on this, the points in the DEM image are projected using the visible light camera as the origin. For example, the projection can be based on any projection algorithm. The position information of each point after projection is supplemented according to the elevation and latitude / longitude information, and the DEM image is converted into a DEM side view.

[0129] The event localization method provided in this application uses multiple second-side visible light images of the scene area to be detected, acquired from different perspectives, to stitch together a side-view panoramic visible light image. This image is then matched with a DEM side view converted from a DEM image. Compared to matching multiple single-view images with the DEM side view separately, this method can reduce the number of matching points, simplify the search for matching points, improve matching accuracy, and provides a more optimized global visualization effect. Converting the DEM image into a DEM side view also reduces the difficulty of obtaining matching points.

[0130] In one embodiment of this application, such as Figure 3 As shown, step S103 above, which obtains the first image position in the first side-view visible light image where a preset type event occurs, and determines the second image position corresponding to the first image position in the side-view panoramic visible light image, includes:

[0131] Step S301: Obtain at least one auxiliary image of the first side-view visible light image, first view information of the first side-view visible light image, and second view information of the auxiliary image;

[0132] The field of view of the auxiliary image includes the field of view of the scene area corresponding to the position of the first image, and the magnification of the auxiliary image is less than the magnification of the first side-view visible light image.

[0133] The first perspective information is the perspective information when the camera captures the first side-view visible light image, and the second perspective information is the perspective information when the camera captures the auxiliary image. The scene area corresponding to the position of the first image is the part of the scene area in the scene area to be detected where a preset type of event occurs.

[0134] An auxiliary image refers to any side-view panoramic visible light image that helps confirm the scene area corresponding to the location of the first image, and the corresponding location of the second image. It has the same field of view as the first side-view visible light image, including the scene area corresponding to the location of the first image. The auxiliary image has a lower magnification than the first side-view visible light image; that is, the auxiliary image contains more scene area than the first side-view visible light image, making it easier to identify the first area. For example, it can be acquired by other devices near the device acquiring the first side-view visible light image, or by a device capable of monitoring a portion of the area where the device acquiring the first side-view visible light image is located.

[0135] In one example, the auxiliary image can be an image with a different magnification at the same viewpoint as the first side-view visible light image, or it can be an image with the same field of view at a different viewpoint as the first side-view visible light image, and with the same viewpoint as a certain image region in the side-view panoramic visible light image where the first side-view visible light image is located.

[0136] Step S302: Determine the position of the third image corresponding to the position of the first image in the auxiliary image based on the first view information and the second view information;

[0137] Step S303: Based on the second perspective information and the third image position, determine the second image position corresponding to the first image position in the side-view panoramic visible light image.

[0138] The event localization method provided in this application embodiment is based on a first side-view visible light image with the same field of view and its corresponding auxiliary image. It determines the position of the scene area corresponding to the first image position in the side-view panoramic visible light image of the scene area to be detected based on multiple viewpoint information, which can achieve the determination of event position more quickly and accurately.

[0139] In one embodiment of this application, the first perspective information is first PTZ information;

[0140] Step 301 above, which involves acquiring at least one auxiliary image of the first side-view visible light image, first viewpoint information of the first side-view visible light image, and second viewpoint information of the auxiliary image, includes:

[0141] Obtain N auxiliary images of the first side-view visible light image, the first viewpoint information of the first side-view visible light image, and the PTZ information of the auxiliary images, wherein N is an integer greater than 1, the P and T information of the first side-view visible light image is the same as that of the first N-1 auxiliary images, the magnification of the N-1th auxiliary image is the same as that of the Nth auxiliary image, the magnification of the 1st auxiliary image is greater than that of the first side-view visible light image, the magnification of the (i+1)th auxiliary image is greater than that of the 1st auxiliary image, i∈[1, N-1], and i is an integer.

[0142] PTZ information refers to the left / right / up / down movement of the acquisition device, lens zoom, and zoom control information, which are the perspective information of the image.

[0143] For example, when the first viewpoint information is P=0, T=5, Z=50, and the image with viewpoint information P=0, T=0, Z=1 has a common field of view (common view) with the first side-view visible light image, then the images with viewpoint information of viewpoint 1 (P=0, T=5, Z=25), viewpoint 2 (P=0, T=5, Z=12), viewpoint 3 (P=0, T=5, Z=6), viewpoint 4 (P=0, T=5, Z=3), viewpoint 5 (P=0, T=5, Z=2), viewpoint 6 (P=0, T=5, Z=1), and viewpoint 7 (P=0, T=0, Z=1) can all be auxiliary images, and all of the above viewpoints can be second viewpoint information.

[0144] Step S302 above, based on the first viewpoint information and the second viewpoint information, determines the third image position corresponding to the first image position in the auxiliary image, including:

[0145] Step 1: Based on the first PTZ information and the PTZ information of the first auxiliary image, determine the mapping position of the scene region corresponding to the position of the first image in the first auxiliary image.

[0146] Step 2: Based on the PTZ information of the i-th and i+1-th auxiliary images, determine the mapping position of the scene region corresponding to the first image position in the i+1-th auxiliary image, wherein the third image position is the mapping position of the scene region corresponding to the first image position in the N-th auxiliary image.

[0147] The event localization method provided in this application determines a first side-view visible light image and its corresponding auxiliary image, which include the same field of view, based on the rules of view information. It determines the position of the second image in the side-view panoramic visible light image of the scene area to be detected based on multiple view information, which can achieve the determination of event location more quickly and accurately.

[0148] In one embodiment of this application, such as Figure 4 As shown, the second side-view visible light image includes the first side-view visible light image;

[0149] Step S103 above, which involves obtaining the first image position in the first side-view visible light image where a preset type event occurs, and determining the second image position corresponding to the first image position in the side-view panoramic visible light image, includes:

[0150] Step S401: Obtain the first image position in the first side-view visible light image where a preset type of event occurs, and the coordinate transformation relationship between the image coordinate system of the first side-view visible light image and the image coordinate system of the side-view panoramic visible light image;

[0151] Step S402: According to the coordinate transformation relationship, the first image position is converted into the second image position in the side-view panoramic visible light image.

[0152] In this embodiment, if the first side-view visible light image can clearly show the scene area where a preset type of event occurs, then the first side-view visible light image can also be used as a second side-view visible light image for stitching together a side-view panoramic visible light image. In this case, the coordinate transformation relationship between the image coordinate system of the first side-view visible light image and the image coordinate system of the side-view panoramic visible light image can be directly obtained. In one example, based on feature point matching of the first side-view visible light image and the side-view panoramic visible light image, the rotation matrix, Euler angles, quaternions, etc., between the image coordinate systems of the first side-view visible light image and the side-view panoramic visible light image can be calculated to achieve coordinate transformation between the two image coordinate systems. Based on the coordinate transformation relationship, the position of the first image is converted into the position of the second image in the side-view panoramic visible light image.

[0153] The event localization method provided in this application, when the second side-view visible light image includes the first side-view visible light image, can directly convert the first image position into the second image position in the side-view panoramic visible light image based on the coordinate transformation relationship between the image coordinate system of the first side-view visible light image and the image coordinate system of the side-view panoramic visible light image, thereby improving the efficiency of event localization.

[0154] In one embodiment of this application, such as Figure 5-1As shown, step S105 above performs dense mapping on the side-view visible light image to be matched and the DEM side view based on the scene point matching result to obtain a dense mapping relationship, including:

[0155] Step S501: Divide the side-view visible light image to be matched into multiple visible light regions according to the scene points, and divide the DEM side view into multiple DEM side view regions.

[0156] like Figure 5-2 As shown, the visible light image to be matched is divided into multiple triangular visible light regions, and the DEM side view is divided into multiple triangular DEM side view regions. Specifically, the triangular shape is only an example, and other shapes can also be used for region division, such as rectangles, irregular shapes, etc., as long as the same division is performed on the visible light image to be matched and the DEM side view.

[0157] Step S502: Based on the scene point matching results, calculate the mapping transformation matrix between the visible light region and the DEM side view region that have a matching relationship, and obtain the dense mapping relationship.

[0158] Based on the scene point matching results between the side-view visible light image to be matched and the DEM side view obtained above, this result can represent the sparse mapping between the side-view visible light image to be matched and the DEM side view. Based on this, the mapping transformation matrix between the matching visible light region and the DEM side view region is calculated. For example, it can be an affine transformation, homography transformation, etc. Then, based on the mapping transformation matrix and the coordinates of the scene points in the DEM side view region, the coordinates of each point in the visible light region are calculated to establish a dense mapping relationship between the side-view visible light image to be matched and the DEM side view, such as... Figure 5-3 As shown.

[0159] The event localization method provided in this application calculates the mapping transformation matrix between the visible light area and the side view area of ​​the DEM according to the division of the region, and establishes a dense mapping relationship between the regions one by one, so as to obtain a more complete and accurate dense mapping relationship between the side view visible light image to be matched and the side view of the DEM.

[0160] In one embodiment of this application, the process example of the above-described event location method can be as follows: Figure 5-4As shown, the preset event type is a fire event, and the location of the preset event type is the fire point. The captured image of the fire point is used as the first side-view visible light image, and the coordinates of the fire point are the first image position. Since there is no single side-view visible light image that can clearly show the fire point position, 1-n captured images from adjacent viewpoints of the fire point are used as the second side-view visible light image for panoramic stitching to obtain a side-view panoramic visible light image and the coordinates of the fire point, which is the second image position. The DEM map is converted into a DEM side view based on the camera point information. Scene point matching is performed on the side-view panoramic visible light image and the DEM side view to find matching point pairs and obtain a sequence of matching point pairs to achieve sparse mapping. Dense mapping is performed based on the scene point matching results to obtain the dense mapping relationship between the side-view panoramic visible light image and the DEM side view. Based on the three-dimensional positional relationship of each point in the DEM side view and the dense mapping relationship, the three-dimensional position information corresponding to the coordinates of the fire point in the second image is determined, thus realizing the location of the fire point.

[0161] See Figure 6 This application also provides a schematic diagram of the structure of an event location device, including:

[0162] The event detection module 601 is used to detect preset types of events in the first side-view visible light image of the scene area to be detected, and obtain the event detection results;

[0163] The image acquisition module 602 is used to acquire the side-view visible light image to be matched for the scene area to be detected, and the DEM side view, when the event detection result indicates that there is a preset type of event in the first side-view visible light image. The DEM side view includes the three-dimensional position information of each point in the scene area to be detected.

[0164] The image position determination module 603 is used to obtain the first image position in the first side-view visible light image where a preset type event occurs, and to determine the second image position corresponding to the first image position in the side-view visible light image to be matched.

[0165] Scene point matching module 604 is used to match the same scene points in the side view visible light image to be matched and the DEM side view to obtain scene point matching results;

[0166] The dense mapping module 605 is used to perform dense mapping on the side view visible light image to be matched and the DEM side view based on the scene point matching result, so as to obtain a dense mapping relationship;

[0167] The location information determination module 606 is used to determine the three-dimensional location information corresponding to the second image location based on the three-dimensional location information of each point in the DEM side view and the dense mapping relationship.

[0168] The event localization device provided in this application provides high-precision location information for each scene point by densely mapping the scene points between the side-view visible light image to be matched and the DEM side view. Even at long distances in large scene areas, high positioning accuracy can be achieved if a certain number of matching points are obtained through dense mapping. Based on this, the localization information of the occurrence of a preset type of event is obtained, thereby improving the accuracy of localization information in large scenes. In addition, when the event detection result indicates that a preset type of event exists in the first side-view visible light image, the first image position of the first image in which the preset type of event occurs is used to determine the second image position corresponding to the first image position in the side-view visible light image to be matched. Then, the corresponding three-dimensional position information is determined based on the second image position. Since this is an online event localization process, it can avoid unexpected situations such as camera repositioning errors, increased positioning errors due to changes in camera extrinsic parameters, and calibration data failure that may be caused by offline calibration in advance. This further improves the accuracy of localization information in large scenes and enhances the long-term positioning accuracy and reliability of the solution.

[0169] In one embodiment of this application, the image acquisition module 602 is specifically used for:

[0170] When the event detection result indicates that a preset type of event exists in the first side-view visible light image, second side-view visible light images of the scene area to be detected are acquired from multiple different viewpoints, wherein adjacent viewpoints have overlapping fields of view;

[0171] Based on the acquisition angle of each of the second side-view visible light images, the second side-view visible light images are stitched together in a panoramic view to obtain a side-view panoramic visible light image.

[0172] Obtain the DEM image of the scene area to be detected, wherein the DEM image includes the elevation information and latitude and longitude information of each point in the scene area to be detected;

[0173] A DEM side view is generated based on the elevation and latitude / longitude information of each point in the DEM image, and the elevation and latitude / longitude information of the visible light camera that acquired the second side view visible light image.

[0174] The event localization device provided in this application uses multiple second-side visible light images of the scene area to be detected, acquired from different perspectives, to stitch together a side-view panoramic visible light image. This image is then matched with a DEM side view converted from a DEM image. Compared to matching multiple single-view images separately with the DEM side view, this method increases the number of matching points, simplifies the difficulty of finding matching points, improves matching accuracy, and provides a more optimized global visualization effect. Converting the DEM image into a DEM side view also reduces the difficulty of obtaining matching points.

[0175] In one embodiment of this application, the image position determination module 603 includes:

[0176] An information acquisition submodule is used to acquire at least one auxiliary image of the first side-view visible light image, first view information of the first side-view visible light image, and second view information of the auxiliary image; wherein, the field of view of the auxiliary image includes the field of view of the scene area corresponding to the position of the first image, and the magnification of the auxiliary image is less than the magnification of the first side-view visible light image.

[0177] The third image position determination submodule is used to determine the third image position corresponding to the first image position in the auxiliary image based on the first view information and the second view information.

[0178] The second image position determination submodule is used to determine the second image position corresponding to the first image position in the side-view panoramic visible light image based on the second view information and the third image position.

[0179] The event location device provided in this application embodiment, based on a first side-view visible light image including the same field of view and its corresponding auxiliary image, determines the position of the scene area corresponding to the first image position in the side-view panoramic visible light image of the scene area to be detected according to multiple perspective information, which can achieve the determination of event position more quickly and accurately.

[0180] In one embodiment of this application, the first perspective information is first PTZ information;

[0181] The information acquisition submodule is specifically used for:

[0182] Obtain N auxiliary images of the first side-view visible light image, the first view angle information of the first side-view visible light image, and the PTZ information of the auxiliary images, wherein N is an integer greater than 1, the P and T information of the first side-view visible light image is the same as that of the first N-1 auxiliary images, the magnification of the N-1th auxiliary image is the same as that of the Nth auxiliary image, the magnification of the 1st auxiliary image is greater than that of the first side-view visible light image, the magnification of the (i+1)th auxiliary image is greater than that of the 1st auxiliary image, i∈[1, N-1], and i is an integer;

[0183] The third image position determination submodule is specifically used for:

[0184] Based on the first PTZ information and the PTZ information of the first auxiliary image, determine the mapping position of the scene region corresponding to the position of the first image in the first auxiliary image;

[0185] Based on the PTZ information of the i-th and i+1-th auxiliary images, the mapping position of the scene region corresponding to the first image position in the i+1-th auxiliary image is determined, wherein the third image position is the mapping position of the scene region corresponding to the first image position in the N-th auxiliary image.

[0186] The event location device provided in this application determines a first side-view visible light image and its corresponding auxiliary image, which include the same field of view, based on the rules of view information. It determines the position of the second image in the side-view panoramic visible light image of the scene area to be detected based on multiple view information, which can achieve the determination of event location more quickly and accurately.

[0187] In one embodiment of this application, the second side-view visible light image includes the first side-view visible light image;

[0188] The image position determination module 603 is specifically used for:

[0189] Obtain the location of the first image in the first side-view visible light image where a preset type of event occurs, and the coordinate transformation relationship between the image coordinate system of the first side-view visible light image and the image coordinate system of the side-view panoramic visible light image;

[0190] According to the coordinate transformation relationship, the first image position is converted into the second image position in the side-view panoramic visible light image.

[0191] The event localization device provided in this application embodiment, when the second side-view visible light image includes the first side-view visible light image, can directly convert the first image position into the second image position in the side-view panoramic visible light image based on the coordinate transformation relationship between the image coordinate system of the first side-view visible light image and the image coordinate system of the side-view panoramic visible light image, thereby improving the efficiency of event localization.

[0192] In one embodiment of this application, the dense mapping module 605 is specifically used for:

[0193] The side-view visible light image to be matched is divided into multiple visible light regions according to the scene points, and the DEM side view is divided into multiple DEM side view regions.

[0194] Based on the scene point matching results, the mapping transformation matrix between the visible light region and the side view region of the DEM with matching relationship is calculated to obtain the dense mapping relationship.

[0195] The event localization device provided in this application calculates the mapping transformation matrix between the visible light area and the side view area of ​​the DEM according to the divided area, and establishes a dense mapping relationship between the areas one by one, so as to obtain a more complete and accurate dense mapping relationship between the side view visible light image to be matched and the side view of the DEM.

[0196] This application also provides an electronic device, such as... Figure 7 As shown, it includes:

[0197] Memory 701 is used to store computer programs;

[0198] The processor 702, when executing the program stored in the memory 701, implements any of the event location methods described above.

[0199] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 702, the communication interface, and the memory 701 communicating with each other via the communication bus.

[0200] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0201] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0202] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0203] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0204] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described event location methods.

[0205] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the event location methods described above.

[0206] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0207] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0208] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and electronic device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0209] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An event localization method, characterized in that, include: The first side-view visible light image of the scene area to be detected is used to detect events of a preset type, and the event detection results are obtained. If the event detection result indicates that a preset type of event exists in the first side-view visible light image, a side-view panoramic visible light image of the scene area to be detected and a DEM side view are obtained, wherein the DEM side view includes the three-dimensional position information of each point in the scene area to be detected. Obtain the first image position in the first side-view visible light image where a preset type of event occurs, and determine the second image position corresponding to the first image position in the side-view panoramic visible light image; Match the same scene points in the side-view panoramic visible light image and the DEM side view to obtain scene point matching results; Based on the scene point matching results, dense mapping is performed on the side-view panoramic visible light image and the DEM side view to obtain a dense mapping relationship; Based on the three-dimensional position information of each point in the DEM side view and the dense mapping relationship, determine the three-dimensional position information corresponding to the second image position; The step of obtaining the first image position in the first side-view visible light image where a preset type event occurs, and determining the second image position corresponding to the first image position in the side-view panoramic visible light image, includes: Acquire at least one auxiliary image of the first side-view visible light image, first viewpoint information of the first side-view visible light image, and second viewpoint information of the auxiliary image; wherein the field of view of the auxiliary image includes the field of view of the scene area corresponding to the position of the first image, and the magnification of the auxiliary image is less than the magnification of the first side-view visible light image; determine the third image position corresponding to the position of the first image in the auxiliary image based on the first viewpoint information and the second viewpoint information; determine the second image position corresponding to the position of the first image in the side-view panoramic visible light image based on the second viewpoint information and the third image position.

2. The method according to claim 1, characterized in that, When the event detection result indicates that a preset type of event exists in the first side-view visible light image, the step of acquiring a side-view panoramic visible light image of the scene area to be detected, and a DEM side view, includes: When the event detection result indicates that a preset type of event exists in the first side-view visible light image, second side-view visible light images of the scene area to be detected are acquired from multiple different viewpoints, wherein adjacent viewpoints have overlapping fields of view; Based on the acquisition angle of each of the second side-view visible light images, the second side-view visible light images are stitched together in a panoramic view to obtain a side-view panoramic visible light image. Obtain the DEM image of the scene area to be detected, wherein the DEM image includes the elevation information and latitude and longitude information of each point in the scene area to be detected; A DEM side view is generated based on the elevation and latitude / longitude information of each point in the DEM image, and the elevation and latitude / longitude information of the visible light camera that acquired the second side view visible light image.

3. The method according to claim 1, characterized in that, The first perspective information is the first PTZ information; The step of acquiring at least one auxiliary image of the first side-view visible light image, first viewpoint information of the first side-view visible light image, and second viewpoint information of the auxiliary image includes: Obtain N auxiliary images of the first side-view visible light image, the first view angle information of the first side-view visible light image, and the PTZ information of the auxiliary images, wherein N is an integer greater than 1, the P and T information of the first side-view visible light image is the same as that of the first N-1 auxiliary images, the magnification of the N-1th auxiliary image is the same as that of the Nth auxiliary image, the magnification of the 1st auxiliary image is greater than that of the first side-view visible light image, the magnification of the (i+1)th auxiliary image is greater than that of the 1st auxiliary image, i∈[1, N-1], and i is an integer; Determining the third image position corresponding to the first image position in the auxiliary image based on the first viewpoint information and the second viewpoint information includes: Based on the first PTZ information and the PTZ information of the first auxiliary image, determine the mapping position of the scene region corresponding to the position of the first image in the first auxiliary image; Based on the PTZ information of the i-th and i+1-th auxiliary images, the mapping position of the scene region corresponding to the first image position in the i+1-th auxiliary image is determined, wherein the third image position is the mapping position of the scene region corresponding to the first image position in the N-th auxiliary image.

4. The method according to claim 2, characterized in that, The second side-view visible light image includes the first side-view visible light image; The step of obtaining the first image position in the first side-view visible light image where a preset type event occurs, and determining the second image position corresponding to the first image position in the side-view panoramic visible light image, includes: Obtain the location of the first image in the first side-view visible light image where a preset type of event occurs, and the coordinate transformation relationship between the image coordinate system of the first side-view visible light image and the image coordinate system of the side-view panoramic visible light image; According to the coordinate transformation relationship, the first image position is converted into the second image position in the side-view panoramic visible light image.

5. The method according to claim 1, characterized in that, The step of performing dense mapping on the side-view panoramic visible light image and the DEM side view based on the scene point matching result to obtain a dense mapping relationship includes: The side-view panoramic visible light image is divided into multiple visible light regions according to the scene points, and the DEM side view is divided into multiple DEM side view regions. Based on the scene point matching results, the mapping transformation matrix between the visible light region and the side view region of the DEM with matching relationship is calculated to obtain the dense mapping relationship.

6. An event location device, characterized in that, include: The event detection module is used to detect preset types of events in the first side view visible light image of the scene area to be detected, and obtain the event detection results; The image acquisition submodule is used to acquire a side-view panoramic visible light image of the scene area to be detected and a DEM side view when the event detection result indicates that there is a preset type of event in the first side-view visible light image. The DEM side view includes the three-dimensional position information of each point in the scene area to be detected. The image position determination module is used to obtain the first image position in the first side-view visible light image where a preset type of event occurs, and to determine the second image position corresponding to the first image position in the side-view panoramic visible light image. The scene point matching module is used to match the same scene points in the side-view panoramic visible light image and the DEM side view to obtain scene point matching results. The dense mapping module is used to perform dense mapping on the side-view panoramic visible light image and the DEM side view based on the scene point matching results, so as to obtain a dense mapping relationship; The location information determination module is used to determine the three-dimensional location information corresponding to the second image location based on the three-dimensional location information of each point in the DEM side view and the dense mapping relationship; The image location determination module includes: An information acquisition submodule is used to acquire at least one auxiliary image of the first side-view visible light image, first view information of the first side-view visible light image, and second view information of the auxiliary image; wherein, the field of view of the auxiliary image includes the field of view of the scene area corresponding to the position of the first image, and the magnification of the auxiliary image is less than the magnification of the first side-view visible light image. The third image position determination submodule is used to determine the third image position corresponding to the first image position in the auxiliary image based on the first view information and the second view information. The second image position determination submodule is used to determine the second image position corresponding to the first image position in the side-view panoramic visible light image based on the second view information and the third image position.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-5.

Images