Fire point positioning method and device, equipment and storage medium

Abstract

The application provides a fire point positioning method and device, equipment and a storage medium, relates to the technical field of fire point positioning, and is used for accurately positioning a fire starting point in an initial stage of a fire. The method comprises the following steps: acquiring a first smoke image and a second smoke image of a fire starting point; determining a first smoke highest point and a first smoke lowest point in the first smoke image, and determining a second smoke highest point and a second smoke lowest point in the second smoke image; determining a first ray; determining a second ray; determining a third ray; determining a fourth ray; determining a first intersection point of the first ray and the second ray, and determining a second intersection point of the third ray and the fourth ray; determining a target straight line according to the first intersection point and the second intersection point, and determining a fire starting point position according to the target straight line and a digital elevation model.

Description

Technical Field

[0001] The present application relates to the technical field of fire point positioning, and in particular to a fire point positioning method, device, equipment and storage medium. Background Art

[0002] Areas such as forests and valleys are usually sparsely populated and large in area. As a result, when a fire occurs in forests and valleys, it is difficult to find the geographical location of the fire in the above areas, which makes fire handling difficult and even causes huge losses.

[0003] To facilitate the location of the fire, imaging equipment can be deployed in the aforementioned area to automatically identify fires within the area. Specifically, a fire location algorithm can be used to calculate the location of the fire based on the imaging equipment's geographic location, the current horizontal and elevation angles, and a digital elevation model (DEM).

[0004] However, in reality, the scene is usually accompanied by thick smoke at the beginning of a fire. At this time, as the smoke floats in the air, if the height of the smoke has exceeded the installation height of the monitoring imaging equipment, then the pitch angle of the smoke captured by the imaging equipment will be greater than 0. The height value of the fire point measured by the imaging equipment (i.e., the z value) is biased large. The higher the smoke floats, the greater the z value deviation. This results in the equation for calculating the ray based on the horizontal angle and pitch angle reported by the imaging equipment having no intersection with the digital elevation model in three-dimensional space. Mathematically speaking, these two equations have no solution, and the geographical location of the fire point cannot be calculated. Summary of the Invention

[0005] Based on the above technical problems, the present application provides a fire point positioning method, device, equipment and storage medium for accurately locating the fire point in the initial stage of a fire.

[0006] In a first aspect, the present application provides a fire point locating method, the method comprising: obtaining a first smoke image and a second smoke image of a fire point at the same time; the first smoke image is acquired by a first image acquisition device located at a first observation position, and the second smoke image is acquired by a second image acquisition device located at a second observation position, and the first observation position is different from the second observation position; determining a first smoke point and a second smoke point in the first smoke image, and determining a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image; determining a first pitch angle and a first horizontal angle of the first image acquisition device according to the first smoke point, and taking the first observation position point where the first image acquisition device is located as the ray end point, and taking the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle as the ray direction, determining the first ray; determining a second pitch angle and a second horizontal angle of the second image acquisition device according to the third smoke point, and taking the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle as the ray direction The second ray is determined by taking the second observation position point as the ray endpoint and the acquisition direction of the second image acquisition device at the second pitch angle and the second horizontal angle as the ray direction; according to the second smoke point, the third pitch angle and the third horizontal angle of the first image acquisition device are determined, and the third ray is determined by taking the first observation position point where the first image acquisition device is located as the ray endpoint and the acquisition direction of the first image acquisition device at the third pitch angle and the third horizontal angle as the ray direction; according to the fourth smoke point, the fourth pitch angle and the fourth horizontal angle of the second image acquisition device are determined, and the fourth ray is determined by taking the second observation position point where the second image acquisition device is located as the ray endpoint and the acquisition direction of the second image acquisition device at the fourth pitch angle and the fourth horizontal angle as the ray direction; a first intersection point is determined based on the first ray and the second ray, and a second intersection point is determined based on the third ray and the fourth ray; a target straight line is determined based on the first intersection point and the second intersection point, and the fire point position is determined based on the target straight line and the digital elevation model.

[0007] This application analyzes smoke images taken from multiple observation points, and comprehensively determines the location of the fire point based on the analysis results. Compared with the analysis based on a single observation point in the related art, the reference information of this application is more comprehensive, providing strong support for the subsequent fire point positioning. Specifically, for a smoke image taken from one observation point, this application first determines any two smoke points in the smoke image taken from the observation point, and records them as the first smoke point and the second smoke point. For a smoke image taken from another observation point, this application determines the point corresponding to the position of the first smoke point and the point corresponding to the position of the second smoke point in the smoke image taken from the observation point, and records them as the third smoke point and the fourth smoke point. Furthermore, with the two observation point positions as endpoints, rays are drawn in the direction of the smoke points determined by their respective smoke images, so that each observation point can obtain two rays in different directions. This application uses rays pointing from two different observation points in the direction of any corresponding smoke point to determine an intersection in space (recorded as the first intersection), and uses rays pointing from two different observation points in the direction of another corresponding smoke point to obtain another intersection in space (recorded as the second intersection). It can be understood that two points can determine a straight line, and since the two points are in different positions in space, the straight line determined by the two points can intersect with the ground surface, and then this application can combine the straight line with the digital elevation model to determine the specific location of the fire. In this way, it avoids the situation where, when locating the fire based on the smoke in the initial stage, the smoke is at a higher position, resulting in no intersection with the digital elevation model, and thus the fire point cannot be located.

[0008] In one possible implementation, determining a first smoke point and a second smoke point in a first smoke image includes: acquiring adjacent frame images of the first smoke image captured by a first image acquisition device; determining a changed region in the first smoke image based on the adjacent frame images and the first smoke image; determining the highest point of the changed region as the first smoke point; and determining the lowest point of the changed region as the second smoke point.

[0009] In a possible implementation, determining the first intersection point according to the first ray and the second ray includes: determining an approximate intersection point of the first ray and the second ray, and using the approximate intersection point as the first intersection point.

[0010] In one possible implementation, determining the approximate intersection point of the first ray and the second ray includes: determining a first target point on the first ray that is closest to the second ray, and determining a second target point on the second ray that is closest to the first ray; obtaining a target line segment based on the first target point and the second target point; and determining any point on the target line segment as the approximate intersection point of the first ray and the second ray, or determining a point on the target line segment that satisfies a preset interception ratio as the approximate intersection point.

[0011] In a second aspect, the application provides a fire point positioning method, which comprises: acquiring a first smoke image and a second smoke image of a fire point at the same time; the first smoke image is acquired by a first image acquisition device at a first observation position, and the second smoke image is acquired by a second image acquisition device at a second observation position; the first observation position is different from the second observation position; determining a first smoke point and a second smoke point in the first smoke image, and determining a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image; determining a first spatial coordinate point of the first smoke point, a second spatial coordinate point of the second smoke point, a third spatial coordinate point of the third smoke point, and a fourth spatial coordinate point of the fourth smoke point; in a case where a deviation between the first spatial coordinate point and the third spatial coordinate point is less than or equal to a preset deviation, determining a first target point according to the first smoke point and the third smoke point, and in a case where a deviation between the second spatial coordinate point and the fourth spatial coordinate point is less than or equal to the preset deviation, determining a second target point according to the third smoke point and the fourth smoke point; determining a target straight line according to the first target point and the second target point, and determining a fire point position according to the target straight line and a digital elevation model.

[0012] In a possible implementation, the determining of the first target point according to the first smoke point and the third smoke point comprises: determining the first smoke point as the first target point; or determining the third smoke point as the first target point; or determining a midpoint between the first smoke point and the third smoke point as the first target point; or determining a certain point on a line connecting the first smoke point and the third smoke point as the first target point according to a preset calculation ratio.

[0013] In a possible implementation, the determining of the fire point position according to the target straight line and the digital elevation model comprises: determining an intersection point of the target straight line and the digital elevation model as the fire point position.

[0014] In a third aspect, the present application provides a fire point locating device, which includes an acquisition unit and a determination unit; the acquisition unit is used to acquire a first smoke image and a second smoke image of the fire point at the same time; the first smoke image is acquired by a first image acquisition device located at a first observation position, and the second smoke image is acquired by a second image acquisition device located at a second observation position, and the first observation position is different from the second observation position; the determination unit is used to determine a first smoke point and a second smoke point in the first smoke image, and to determine a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image; the determination unit is also used to determine a first pitch angle and a first horizontal angle of the first image acquisition device according to the first smoke point, and to determine a first ray with the first observation position point where the first image acquisition device is located as the ray end point and the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle as the ray direction; the determination unit is also used to determine a second pitch angle and a second horizontal angle of the second image acquisition device according to the third smoke point, and to determine a second pitch angle and a second horizontal angle of the second image acquisition device according to the second image acquisition device The second observation position point where the image acquisition device is located is used as the ray endpoint, and the acquisition direction of the second image acquisition device at the second pitch angle and the second horizontal angle is used as the ray direction to determine the second ray; the determination unit is further used to determine the third pitch angle and the third horizontal angle of the first image acquisition device based on the second smoke point, and to determine the third ray based on the first observation position point where the first image acquisition device is located as the ray endpoint, and the acquisition direction of the first image acquisition device at the third pitch angle and the third horizontal angle as the ray direction; the determination unit is further used to determine the fourth pitch angle and the fourth horizontal angle of the second image acquisition device based on the fourth smoke point, and to determine the fourth ray based on the second observation position point where the second image acquisition device is located as the ray endpoint, and the acquisition direction of the second image acquisition device at the fourth pitch angle and the fourth horizontal angle as the ray direction; the determination unit is further used to determine a first intersection point based on the first ray and the second ray, and to determine a second intersection point based on the third ray and the fourth ray; the determination unit is further used to determine a target line based on the first intersection point and the second intersection point, and to determine the location of the fire point based on the target line and the digital elevation model.

[0015] In one possible implementation, the determination unit is specifically used to: obtain adjacent frame images of a first smoke image captured by a first image acquisition device; determine a changed area in the first smoke image based on the adjacent frame images and the first smoke image; determine the highest point of the changed area as the first smoke point; and determine the lowest point of the changed area as the second smoke point.

[0016] In a possible implementation, the determining unit is specifically configured to determine an approximate intersection point between the first ray and the second ray, and use the approximate intersection point as the first intersection point.

[0017] In one possible implementation, the determination unit is specifically used to: determine a first target point on the first ray that is closest to the second ray, and determine a second target point on the second ray that is closest to the first ray; obtain a target line segment based on the first target point and the second target point; use any point on the target line segment as an approximate intersection point between the first ray and the second ray, or determine a point on the target line segment that meets a preset interception ratio as an approximate intersection point.

[0018] In a fourth aspect, the present application provides a fire point locating device, which includes an acquisition unit and a determination unit; the acquisition unit is used to acquire a first smoke image and a second smoke image of the fire point at the same time; the first smoke image is acquired by a first image acquisition device located at a first observation position, and the second smoke image is acquired by a second image acquisition device located at a second observation position, and the first observation position is different from the second observation position; the determination unit is used to determine a first smoke point and a second smoke point in the first smoke image, and to determine a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image; the determination unit is also used to determine the first smoke point The first spatial coordinate point of the fog point, the second spatial coordinate point of the second smoke point, the third spatial coordinate point of the third smoke point, and the fourth spatial coordinate point of the fourth smoke point; the determination unit is further used to determine the first target point according to the first smoke point and the third smoke point when the deviation between the first spatial coordinate point and the third spatial coordinate point is less than or equal to the preset deviation, and to determine the second target point according to the third smoke point and the fourth smoke point when the deviation between the second spatial coordinate point and the fourth spatial coordinate point is less than or equal to the preset deviation; the determination unit is further used to determine the target straight line according to the first target point and the second target point, and determine the position of the fire point according to the target straight line and the digital elevation model.

[0019] In one possible implementation, the determination unit is specifically used to: determine the first smoke point as the first target point; or, determine the third smoke point as the first target point; or, determine the midpoint between the first smoke point and the third smoke point as the first target point; or, based on a preset calculation ratio, determine a point on the line connecting the first smoke point and the third smoke point as the first target point.

[0020] In a possible implementation, the determining unit is specifically configured to determine an intersection of the target straight line and the digital elevation model as the location of the fire starting point.

[0021] In a fifth aspect, the present application provides an electronic device comprising a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method described in the first aspect and / or the second aspect above.

[0022] In a sixth aspect, the present application provides a fire point positioning system, comprising the electronic device of the fifth aspect and a plurality of image acquisition devices.

[0023] In a seventh aspect, the present application provides a computer program product, which, when running in an electronic device, causes the electronic device to perform the method of the first aspect, so as to implement the method of the first aspect and / or the second aspect.

[0024] In an eighth aspect, the present application provides a readable storage medium, which comprises: software instructions; and when the software instructions run in an electronic device, the electronic device implements the method of the first aspect and / or the second aspect.

[0025] The advantages of the second aspect to the eighth aspect can refer to the first aspect, and will not be described again. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1 A digital elevation model diagram of a certain area provided by the embodiments of the present application;

[0028] Figure 2 A smoke scene diagram at the beginning of the fire provided by the embodiments of the present application;

[0029] Figure 3 A fire point positioning diagram of related technology provided by the embodiments of the present application;

[0030] Figure 4 A composition diagram of a fire point positioning system provided by the embodiments of the present application;

[0031] Figure 5 A composition diagram of an electronic device provided by the embodiments of the present application;

[0032] Figure 6 A flow diagram of a fire point positioning method provided by the embodiments of the present application;

[0033] Figure 7 Two consecutive smoke images provided by the embodiments of the present application;

[0034] Figure 8 A smoke area diagram provided by the embodiments of the present application;

[0035] Figure 9 A schematic diagram of the highest point of smoke provided in an embodiment of the present application;

[0036] Figure 10 A schematic diagram of fire point positioning provided in an embodiment of the present application;

[0037] Figure 11 A schematic diagram of image stereo correction provided in an embodiment of the present application;

[0038] Figure 12 A schematic diagram of the composition of the fire point locating device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to enable ordinary people in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0040] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0041] In addition, in the description of the embodiments of this application, unless otherwise specified, " / " means or. For example, A / B can mean A or B. "And / or" in this article is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "plurality" means two or more than two.

[0042] Before explaining the embodiments of the present application in detail, some relevant terms and related technologies involved in the embodiments of the present application are first introduced.

[0043] Elevation: refers to the distance from a point to the base surface along the plumb line.

[0044] Digital Elevation Model (DEM): It is a digital simulation of the ground terrain (that is, a digital expression of the terrain surface morphology) through limited terrain elevation data. It is a physical ground model that represents the ground elevation in the form of a set of ordered numerical arrays.

[0045] Figure 1 The digital elevation model of a certain area is shown, through which the elevation changes of different locations on the surface of the area can be expressed.

[0046] Horizontal angle: This is the dihedral angle formed by projecting the directional lines from the survey station to the two targets onto the horizontal plane. During measurement, the actual observed angle on the ground is projected onto the horizontal dial of the goniometer, and the horizontal angle value is calculated based on the dial reading. The horizontal angle is measured clockwise from 0 to 360 degrees on the horizontal plane.

[0047] Pitch angle: This refers to the angle of an object or observer relative to the horizontal plane. In photography and videography, pitch angle is used to describe the degree to which a camera or camcorder lens is tilted relative to the horizontal plane.

[0048] Parallax refers to the difference in direction when observing the same object from two points at a certain distance. The angle between the two points as seen from the object is called the parallax angle, and the line connecting the two points is called the baseline. Knowing the parallax angle and the length of the baseline allows calculation of the distance between the object and the observer.

[0049] Areas like forests and suburbs are often sparsely populated and large, making it difficult to quickly and accurately locate the fire source when a fire breaks out, thus delaying fire response. For example, locating and extinguishing a fire five minutes after starting is easier and results in less damage than locating and extinguishing it 25 minutes after starting. As time passes, the fire may grow larger, making extinguishing more difficult and damaging. Therefore, locating the fire source as early as possible can reduce both damage and the difficulty of extinguishing the fire.

[0050] like Figure 2 As shown, at the beginning of a fire, the intensity is small and the combustion of materials is usually incomplete. This stage is often accompanied by thick smoke. As the smoke rises into the air, the monitoring imaging equipment can detect it and report the horizontal and elevation angles of the smoke. The horizontal and elevation angles reported by the equipment, combined with the digital elevation model, can be used to locate the fire point.

[0051] Figure 3 This is a spatial diagram of the horizontal angle and pitch angle in the fire point positioning according to the related technology of this application. Figure 3 Point A is the location of the imaging equipment (e.g., thermal imaging equipment) used to find the fire point. The imaging equipment can be installed on the observation tower, so that it has a certain height, which is more conducive to finding the fire point. Figure 3 The middle T point is the ignition point. When the ignition point appears, the imaging device rotates under the drive of the pan / tilt head so that the ignition point appears at the center of the lens of the imaging device and coincides with the optical axis of the lens. Figure 3The pitch angle β and horizontal angle α in the image reflect the positional relationship between the imaging device and the fire point. For example, related technologies can calculate the ray equation from point A to point T, and then combine the ray equation with the digital elevation model to obtain the location coordinates of the fire point.

[0052] However, in actual applications, the scene is usually accompanied by thick smoke at the beginning of a fire. At this time, as the smoke floats in the air, if the height of the smoke has exceeded the installation height of the monitoring imaging equipment, then the pitch angle of the smoke captured by the imaging equipment will be greater than 0. The height value of the fire point measured by the imaging equipment (i.e., the z value) is biased large. The higher the smoke floats, the greater the z value deviation. This results in the equation for calculating the ray based on the horizontal angle and pitch angle reported by the imaging equipment having no intersection with the digital elevation model of the three-dimensional space. Mathematically speaking, these two equations have no solution, and the geographical location of the fire point cannot be calculated.

[0053] It can be seen that the relevant technology cannot effectively solve the deviation caused by smoke when a fire just starts (the above phenomenon is more obvious in hilly and mountainous terrain). It is easy to fail to locate the fire point in the initial stage of the fire. When the fire is larger, the fire point may be located, but the losses and difficulty of fire extinguishing will increase at this time.

[0054] In view of the above problems, an embodiment of the present application provides a fire point locating method, which analyzes smoke images taken from multiple observation points, finds the target straight line corresponding to the fire point position, and combines the target straight line with the digital elevation model to determine the specific fire point position, so as to avoid the situation in which the fire point position cannot be located due to the high position of the smoke when locating the fire based on the smoke in the initial stage.

[0055] The fire point locating method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0056] The fire point locating method provided in the embodiment of the present application can be applied to the fire point locating system. Figure 4 Figure 2 shows a structural diagram of the fire point positioning system. Figure 4 As shown, the fire point locating system 10 includes a plurality of image acquisition devices 11 and a fire point locating device 12. The image acquisition devices 11 and the fire point locating devices 12 can be connected via a wired network or a wireless network.

[0057] The image acquisition device 11 can be used to acquire images of the fire point (eg, smoke images of the fire point).

[0058] The image acquisition device 11 may be an industrial camera, a camera installed at a specific location, an internet protocol camera (IPC), an imaging device (eg, a thermal imaging device), or the like.

[0059] Optionally, each image acquisition device 11 can be deployed at a different observation position to avoid blind spots, provide more comprehensive reference information, and provide strong support for subsequent fire location. For example, the multiple image acquisition devices 11 include a first image acquisition device and a second image acquisition device, wherein the first image acquisition device is located at a first observation position and the second image acquisition device is located at a second observation position, and the first observation position and the second observation position are different.

[0060] In some embodiments, the image acquisition device 11 may also be used to send the acquired images to the fire point locating device 12 .

[0061] As described above, the image acquisition device 11 and the fire point locating device 12 may be connected via a wired network or a wireless network. The wired network or the wireless network may include one or more media or devices capable of transmitting image data from the image acquisition device 11 to the fire point locating device 12.

[0062] In some embodiments, the wired or wireless network may include one or more communication media that enable the image acquisition device 11 to send image data directly to the fire point locating device 12 in real time. In this embodiment, the image acquisition device 11 may modulate the image data according to a communication standard (e.g., a wireless communication protocol) and send the modulated image data to the fire point locating device 12. The one or more communication media may include wireless, and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. Optionally, the one or more communication media may form part of a packet-based network, which may be, for example, a local area network, a wide area network, or a global network (e.g., the Internet). Optionally, the one or more communication media may also include routers, switches, base stations, or other devices that facilitate communication from the image acquisition device 11 to the fire point locating device 12.

[0063] The fire point positioning device 12 can be used to analyze the smoke images collected by each image acquisition device 11, and comprehensively determine the fire point position based on the analysis results. The specific positioning process can refer to the fire point positioning method introduced in the following method embodiment, which will not be repeated here.

[0064] The fire point locating device 12 can be an electronic device with computing and processing capabilities, such as a computer or a server.

[0065] The server can be a single server or a server cluster formed by multiple servers. In some embodiments, the server cluster can also be a distributed cluster. Optionally, the server can also be implemented on a cloud platform, for example, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, and the like, or any combination thereof. The embodiments of the present application do not make any limitation in this regard.

[0066] It should be noted that the above Figure 4 The image acquisition device 11 and the fire point positioning device 12 are taken as independent devices for example. Optionally, the image acquisition device 11 and the fire point positioning device 12 can also be combined into one device. For example, the image acquisition device 11 or its corresponding function, and the fire point positioning device 12 or its corresponding function can be integrated into one device. The embodiments of the present application do not make any limitation in this regard.

[0067] The execution subject of the fire point positioning method provided by the embodiments of the present application can be the fire point positioning device 12 described above. As described above, the fire point positioning device 12 can be a computer or a server or other electronic device with computing and processing functions. Optionally, the fire point positioning device 12 can also be a processor (such as a central processing unit (CPU)) in the foregoing electronic device; or the fire point positioning device 12 can also be an application (APP) installed in the foregoing electronic device with a passenger behavior detection function; or the fire point positioning device 12 can also be a functional module in the foregoing electronic device with a passenger behavior detection function, and the like. The embodiments of the present application do not make any limitation in this regard.

[0068] For the sake of simplicity, the following will be described by taking the fire point positioning device 12 as an example of an electronic device.

[0069] Figure 5 The composition of the electronic device provided by the embodiments of the present application is shown in the following schematic diagram. As shown in the schematic diagram, the electronic device can include a processor 20, a memory 21, a communication line 22, a communication interface 23, and an input / output interface 24. Figure 5

[0070] The processor 20, the memory 21, the communication interface 23, and the input / output interface 24 can be connected through the communication line 22.

[0071] ​The processor 20 is used to execute the instructions stored in the memory 21 to implement the fault analysis method provided in the following embodiments of the present application. The processor 20 can be a CPU, a general-purpose processor network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller (MCU) / single-chip microcomputer, a programmable logic device (PLD), or any combination thereof. The processor 20 can also be any other device with processing functions, such as a circuit, a device, or a software module, which is not limited in the embodiments of the present application. In one example, the processor 20 may include one or more CPUs, such as Figure 5 As an optional implementation, the electronic device may include multiple processors, for example, in addition to the processor 20, it may also include a processor 25 ( Figure 5 The dashed line is used as an example.

[0072] Memory 21 is used to store instructions. For example, the instruction can be a computer program. Optionally, the memory 21 can be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, etc., and the embodiments of the present application are not limited to this.

[0073] It should be noted that the memory 21 may exist independently of the processor 20 or may be integrated with the processor 20. The memory 21 may be located inside the electronic device or outside the electronic device, which is not limited in the embodiment of the present application.

[0074] The communication line 22 is used to transmit information between the various components included in the electronic device.

[0075] Communication interface 23 is used to communicate with other devices (e.g., the image acquisition device 11) or other communication networks. Such other communication networks may be Ethernet, radio access networks (RAN), wireless local area networks (WLAN), etc. Communication interface 23 may be a module, circuit, transceiver, or any other device capable of communication.

[0076] The input / output interface 24 is used to implement human-computer interaction between a user and the electronic device, for example, to implement action interaction or information interaction between the user and the electronic device.

[0077] For example, the input / output interface 24 may be a mouse, keyboard, display screen, or touch screen screen, etc. Action interaction or information interaction between a user and the electronic device may be achieved through the mouse, keyboard, display screen, or touch screen screen, etc.

[0078] It should be noted that Figure 5 The structure shown in the figure does not constitute a limitation on the electronic device, except Figure 5 In addition to the components shown, the electronic device may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components.

[0079] The following introduces the fire point positioning method provided in the embodiments of the present application.

[0080] Figure 6 The flowchart of the fire point location method provided in the embodiment of the present application is shown in FIG. Figure 5 The electronic device of the hardware structure shown is executed as Figure 6 As shown, the method includes S301 to S308.

[0081] S301: Acquire a first smoke image and a second smoke image of a fire point at the same time.

[0082] The first smoke image is captured by a first image acquisition device located at a first observation position, and the second smoke image is captured by a second image acquisition device located at a second observation position. The first observation position and the second observation position are different.

[0083] As a possible implementation, multiple image acquisition devices can be deployed for any fire point detection area to detect fire points in that area. When a fire occurs in that area, these image acquisition devices can capture smoke images of the fire point and send the captured smoke images to the electronic device. Accordingly, the electronic device can receive a first smoke image captured by a first image acquisition device located at a first observation position, and a second smoke image captured by a second image acquisition device located at a second observation position, and then combine the first and second smoke images to locate the fire point.

[0084] It should be noted that both the first smoke image and the second smoke image can reflect the smoke situation at the fire point. The difference is that the first smoke image and the second smoke image reflect the smoke images at different angles.

[0085] For example, for forest area A, when a fire occurs in the forest area, the first smoke image can be an image of forest area A taken by a first image acquisition device deployed on the east side of forest area A, and the second smoke image can be an image of forest area A taken by a second image acquisition device deployed on the south side of forest area A.

[0086] S302: Determine a first smoke point and a second smoke point in the first smoke image, and determine a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image.

[0087] As a possible implementation, the electronic device may determine a smoke point at any position in the first smoke image as the first smoke point, and locate a smoke point in the second smoke image corresponding to the position of the first smoke point as the third smoke point. Similarly, the electronic device may determine a smoke point at any position other than the first smoke point in the first smoke image as the second smoke point, and locate a fourth smoke point in the second smoke image corresponding to the position of the first smoke point.

[0088] For example, the first smoke point may be the highest point of smoke in the fire detection area captured by the first image acquisition device at the first time point, and the third smoke point may be the highest point of smoke in the fire detection area captured by the second image acquisition device at the first time point. The second smoke point may be the lowest point of smoke in the fire detection area captured by the first image acquisition device at the first time point, and the fourth smoke point may be the lowest point of smoke in the fire detection area captured by the second image acquisition device at the first time point.

[0089] For another example, the first smoke point can be the midpoint of the smoke in the fire detection area captured by the first image capture device at a first time point, while the third smoke point can be the middle of the smoke in the fire detection area captured by the second image capture device at the first time point. The second smoke point can be the lowest point of the smoke in the fire detection area captured by the first image capture device at the first time point, while the fourth smoke point can be the lowest point of the smoke in the fire detection area captured by the second image capture device at the first time point.

[0090] As another possible implementation, the electronic device may first determine a first smoke region in the first smoke image, and then determine the first smoke peak and the first smoke valley based on the highest and lowest points of the first smoke region, with the first smoke peak serving as the first smoke point and the first smoke valley serving as the second smoke point. Similarly, the electronic device may determine a second smoke region in the second smoke image, and then determine the second smoke peak and the second smoke valley based on the highest and lowest points of the second smoke region, with the second smoke peak serving as the third smoke point and the second smoke valley serving as the fourth smoke point.

[0091] Optionally, the electronic device may use the moving area in the smoke image as the smoke area, and determine the highest point and the lowest point of the smoke in the moving area.

[0092] For example, Figure 7 Two consecutive frames of smoke images captured by a first image acquisition device are shown. The electronic device can combine the optical flow method and frame difference method of computer video detection technology to obtain the area of ​​smoke motion (i.e., the changing area) in the video. Specifically, the electronic device can obtain adjacent frames of the first smoke image captured by the first image acquisition device and determine the moving area in the first smoke image based on the adjacent frames and the first smoke image. Furthermore, the electronic device determines the highest point of the moving area as the highest point of the first smoke, and the lowest point of the moving area as the lowest point of the first smoke.

[0093] Similarly, the electronic device can obtain adjacent frames of the second smoke image captured by the second image capture device and determine the motion region in the second smoke image based on the adjacent frames and the second smoke image. Furthermore, the electronic device determines the highest point of the motion region as the highest point of the second smoke image, and the lowest point of the motion region as the lowest point of the second smoke image.

[0094] It is understandable that smoke is usually in constant motion. The motion area obtained by differentially determining this characteristic in this application contains information about the moving target, and can successfully detect the smoke area in the smoke image, thereby accurately determining the highest and lowest points of the smoke.

[0095] In some embodiments, the electronic device can detect the moving area in the image by the above method, but there may still be some noise. In order to eliminate the noise, the electronic device can use the opening operation to eliminate the isolated noise points in the differential image, leaving the moving target information. After the opening operation, the differential image generally cannot contain the complete moving target information, so the electronic device can perform a dilation operation on it to connect the target area, so as to obtain Figure 8 The target shape shown is the shape of the smoke area.

[0096] S303: Determine a first pitch angle and a first horizontal angle of the first image acquisition device based on the first smoke point, and determine a first ray with the first observation position of the first image acquisition device as the ray endpoint and the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle as the ray direction.

[0097] As one possible implementation, the electronic device can control the first image acquisition device to adjust its observation angle until the highest point of the first smoke is located at the center of the first smoke image, thereby obtaining a first pitch angle and a first horizontal angle of the first image acquisition device. Furthermore, the electronic device determines the first ray using the first observation position of the first image acquisition device as the endpoint of a ray and the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle (i.e., the shooting direction of the first image acquisition device) as the ray direction.

[0098] For example, the electronic device is based on Figure 8 In the smoke area, the highest point of the smoke can be determined (i.e. Figure 8 The highest point of the white area in the middle), we get Figure 9 The electronic device can adjust the pitch angle and horizontal angle of the observation device (such as the first image acquisition device) so that S1 is at the center of the picture, record the pitch angle and horizontal angle of the first image acquisition device at this time, and obtain the first pitch angle and the first horizontal angle.

[0099] It can be understood that for a smoke image captured by any image acquisition device, after determining the highest point of the smoke, the electronic device adjusts the horizontal angle and pitch angle of the image acquisition device so that the lens is aimed at the highest point of the smoke (that is, the highest point of the smoke appears in the center of the picture), and then the line between the image acquisition device and the highest point of the smoke can be determined.

[0100] In some embodiments, the electronic device can establish a three-dimensional coordinate system for the fire detection area based on latitude, longitude, and altitude. Since the position of the first image acquisition device is known, the electronic device can establish a ray equation in the three-dimensional coordinate system, using the coordinates of the first image acquisition device in the three-dimensional coordinate system as endpoints and the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle as the ray direction, to obtain the first ray.

[0101] S304. Determine a second pitch angle and a second horizontal angle of the second image acquisition device based on the third smoke point, and determine a second ray with the second observation position of the second image acquisition device as the ray endpoint and the acquisition direction of the second image acquisition device at the second pitch angle and the second horizontal angle as the ray direction.

[0102] As a possible implementation, the electronic device can control the second image acquisition device to adjust its observation angle until the highest point of the second smoke is located at the center of the second smoke image, thereby obtaining a second pitch angle and a second horizontal angle of the second image acquisition device. Furthermore, the electronic device determines the second ray using the second observation position of the second image acquisition device as the endpoint of the ray and the acquisition direction of the second image acquisition device at the second pitch angle and the second horizontal angle (i.e., the shooting direction of the second image acquisition device) as the ray direction.

[0103] In some embodiments, the electronic device can establish a three-dimensional coordinate system for the fire detection area based on latitude, longitude, and altitude. Since the position of the second image acquisition device is known, the electronic device can establish a ray equation in the three-dimensional coordinate system, using the coordinates of the second image acquisition device in the three-dimensional coordinate system as endpoints and the acquisition direction of the second image acquisition device at the second pitch angle and the second horizontal angle as the ray direction, to obtain the second ray.

[0104] S305. Determine a third pitch angle and a third horizontal angle of the first image acquisition device based on the second smoke point, and determine a third ray with the first observation position of the first image acquisition device as the ray endpoint and the acquisition direction of the first image acquisition device at the third pitch angle and the third horizontal angle as the ray direction.

[0105] As one possible implementation, the electronic device may control the first image acquisition device to adjust its observation angle until the lowest point of the first smoke is located at the center of the first smoke image, thereby obtaining a third pitch angle and a third horizontal angle of the first image acquisition device. Furthermore, the electronic device determines a third ray using the first observation position of the first image acquisition device as the endpoint of a ray and the acquisition direction of the first image acquisition device at the third pitch angle and the third horizontal angle (i.e., the shooting direction of the first image acquisition device) as the ray direction.

[0106] In some embodiments, the electronic device can establish a three-dimensional coordinate system for the fire detection area based on latitude, longitude, and altitude. Since the position of the first image acquisition device is known, the electronic device can establish a ray equation in the three-dimensional coordinate system, using the coordinates of the first image acquisition device in the three-dimensional coordinate system as endpoints and the acquisition direction of the first image acquisition device at a third pitch angle and a third horizontal angle as a ray direction, to obtain a third ray.

[0107] S306. Determine a fourth pitch angle and a fourth horizontal angle of the second image acquisition device based on the fourth smoke point, and determine a fourth ray with the second observation position of the second image acquisition device as the ray endpoint and the acquisition direction of the second image acquisition device at the fourth pitch angle and the fourth horizontal angle as the ray direction.

[0108] As a possible implementation, the electronic device may control the second image acquisition device to adjust its observation angle until the lowest point of the second smoke is located at the center of the second smoke image, thereby obtaining a fourth pitch angle and a fourth horizontal angle of the second image acquisition device. Furthermore, the electronic device determines the fourth ray using the second observation position of the second image acquisition device as the endpoint of a ray and the acquisition direction of the second image acquisition device at the fourth pitch angle and the fourth horizontal angle (i.e., the shooting direction of the second image acquisition device) as the ray direction.

[0109] In some embodiments, the electronic device can establish a three-dimensional coordinate system for the fire detection area based on latitude, longitude, and altitude. Since the position of the second image acquisition device is known, the electronic device can establish a ray equation in the three-dimensional coordinate system using the coordinates of the second image acquisition device in the three-dimensional coordinate system as endpoints and the acquisition direction of the second image acquisition device at the fourth pitch angle and the fourth horizontal angle as the ray direction to obtain the fourth ray.

[0110] S307 : Determine a first intersection point based on the first ray and the second ray, and determine a second intersection point based on the intersection point of the third ray and the fourth ray.

[0111] In some embodiments, when the first smoke point and the third smoke point are the same spatial coordinate point, the first ray and the second ray have an intersection in space, and the electronic device may determine the intersection of the first and second rays as the first intersection point. Similarly, when the second smoke point and the fourth smoke point are the same spatial coordinate point, the third ray and the fourth ray have an intersection in space, and the electronic device may determine the intersection of the third and fourth rays as the second intersection point.

[0112] In other embodiments, when the first smoke point and the third smoke point are at different spatial coordinate points, the electronic device may determine an approximate intersection point between the first ray and the second ray, and use this approximate intersection point as the first intersection point. Similarly, when the second smoke point and the fourth smoke point are at different spatial coordinate points, the electronic device may determine an approximate intersection point between the third ray and the fourth ray, and use this approximate intersection point as the second intersection point.

[0113] For example, the electronic device may determine the first target point on the first ray that is closest to the second ray, and determine the second target point on the second ray that is closest to the first ray. Furthermore, the first and second target points are connected to form a target line segment. The electronic device then uses any point on the target line segment as the approximate intersection point between the first and second rays, or any point on the target line segment that satisfies a preset interception ratio as the approximate intersection point. It should be noted that if the deviation between the spatial coordinates corresponding to the first smoke point and the spatial coordinates corresponding to the third smoke point is less than or equal to a preset deviation, the first and third smoke points are considered to be the same spatial coordinate point. In this case, the electronic device may determine the approximate intersection point between the first and second rays based on the spatial coordinates corresponding to the first smoke point, the first ray equation, the spatial coordinates corresponding to the third smoke point, and the second ray equation, and use this approximate intersection point as the first intersection point. Similarly, if the deviation between the spatial coordinates corresponding to the second smoke point and the spatial coordinates corresponding to the fourth smoke point is less than or equal to the preset deviation, the second and fourth smoke points are considered to be the same spatial coordinate point. In this case, the electronic device can determine the approximate intersection of the third ray and the fourth ray based on the spatial coordinates corresponding to the second smoke point, the third ray equation, the spatial coordinates corresponding to the fourth smoke point, and the fourth ray equation, and use the approximate intersection as the second intersection.

[0114] As a possible implementation, if the deviation between the spatial coordinates corresponding to the first smoke point and the spatial coordinates corresponding to the third smoke point is zero, the electronic device can simultaneously solve the equations for the first ray and the second ray to obtain the intersection of the first and second rays, i.e., the first intersection point. Similarly, the electronic device can simultaneously solve the equations for the third ray and the fourth ray to obtain the intersection of the third and fourth rays, i.e., the second intersection point.

[0115] As another possible implementation, if the deviation between the spatial coordinates corresponding to the first smoke point and the spatial coordinates corresponding to the third smoke point is greater than zero but less than or equal to a preset deviation, the electronic device may first determine the projections of the first ray and the second ray on a spatial plane in three-dimensional space, wherein the projection of the first ray on the spatial plane is recorded as the first ray projection, and the projection of the second ray on the spatial plane is recorded as the second ray projection. Furthermore, the electronic device may use the spatial coordinates corresponding to the intersection or coincidence point of the first ray projection and the second ray projection as the first intersection point. Similarly, the electronic device may determine the projections of the third ray and the fourth ray on a spatial plane in three-dimensional space, wherein the projection of the third ray on the spatial plane is recorded as the third ray projection, and the projection of the fourth ray on the spatial plane is recorded as the fourth ray projection. Furthermore, the electronic device may use the spatial coordinates corresponding to the intersection or coincidence point of the third ray projection and the fourth ray projection as the second intersection point.

[0116] As another possible implementation, if the deviation between the spatial coordinates corresponding to the first smoke point and the spatial coordinates corresponding to the third smoke point is greater than zero but less than or equal to a preset deviation (the spatial coordinates corresponding to the first smoke point can be recorded as the first spatial coordinate, and the spatial coordinates corresponding to the third smoke point can be recorded as the third spatial coordinate), the electronic device can further determine the spatial coordinates of the midpoint between the first spatial coordinates and the third spatial coordinates as the intersection of the first ray and the second ray, i.e., the first intersection point. Similarly, the electronic device can further determine the spatial coordinates of the midpoint between the second spatial coordinates and the fourth spatial coordinates as the intersection of the third ray and the fourth ray, i.e., the second intersection point.

[0117] In some embodiments, the electronic device may further first determine whether the first and second peak smoke points correspond to the same three-dimensional coordinate point. If the first and second peak smoke points are the same three-dimensional coordinate point, the electronic device determines that the first and second rays intersect, and determines the intersection of the first and second rays as the first intersection point. If the first and second peak smoke points are not the same three-dimensional coordinate point, the electronic device updates the first and / or second peak smoke points until the first and second peak smoke points are determined to be the same three-dimensional coordinate point. Updating the first and / or second peak smoke points may include: reacquiring a smoke image captured by a first image acquisition device, using the smoke image as the first smoke image, and re-determining the first peak smoke point. Alternatively, updating the first and / or second peak smoke points may include: reacquiring a smoke image captured by a second image acquisition device, using the smoke image as the second smoke image, and re-determining the second peak smoke point.

[0118] It can be understood that when it is determined that the highest (or lowest) point of the smoke observed by two observation points (such as the first image acquisition device and the second image acquisition device) is consistent, the intersection formed by the rays corresponding to the two observation points can be guaranteed to be more accurate.

[0119] In one design, if the deviation between the first spatial coordinate point and the third spatial coordinate point is less than or equal to a preset deviation, the electronic device can also determine a first target point based on the first and third smoke points. This first target point is equivalent to the first intersection point. Similarly, if the deviation between the second and fourth spatial coordinate points is less than or equal to the preset deviation, the electronic device can also determine a second target point based on the second and fourth smoke points. This second target point is equivalent to the second intersection point. Furthermore, the electronic device can determine a target line based on the first and second target points, and determine the location of the fire based on the target line and the digital elevation model.

[0120] There are many ways to determine the first target point and the second target point in the embodiments of the present application. For example, the electronic device can determine the first smoke point as the first target point, or determine the third smoke point as the first target point, or determine the midpoint between the first smoke point and the third smoke point as the first target point, or determine a point on the line connecting the first smoke point and the third smoke point as the first target point according to a preset calculation ratio (such as 3:7).

[0121] S308: Determine a target line according to the first intersection point and the second intersection point, and determine the location of the fire point according to the target line and the digital elevation model.

[0122] As a possible implementation, the electronic device may determine the line connecting the first intersection point and the second intersection point as the target line. Further, the electronic device may determine the intersection of the target line and the digital elevation model as the fire starting point.

[0123] like Figure 10 As shown in the figure, it is a digital elevation map of a hilly area, where point P1 is the deployment position of the first image acquisition device, point P2 is the deployment position of the second image acquisition device, point S1 is the first intersection point (the intersection of the ray from P1 to the highest point of the smoke and the ray from P2 to the highest point of the smoke), point S2 is the second intersection point (the intersection of the ray from P1 to the lowest point of the smoke and the ray from P2 to the lowest point of the smoke), and point F, the intersection point of the straight line between S1 and S2 and the digital elevation map, is the fire starting point.

[0124] In some embodiments, the electronic device can also combine the smoke images of the fire starting point collected by multiple image acquisition devices (three or more image acquisition devices) to comprehensively determine the location of the fire starting point. For example, the electronic device uses three image acquisition devices to obtain the smoke images of the fire starting point collected by the three image acquisition devices. For any two image acquisition devices, the electronic device can determine the intersection point at the highest smoke point and the intersection point at the lowest smoke point through the process of the above embodiment. For three image acquisition devices, the electronic device can determine three intersection points, and the electronic device can fit a straight line based on these three intersection points, and combine the straight line with the digital elevation model to determine the location of the fire point.

[0125] This application analyzes smoke images taken from multiple observation points, and comprehensively determines the location of the fire point based on the analysis results. Compared with the analysis based on a single observation point in the related art, the reference information of this application is more comprehensive, providing strong support for the subsequent fire point positioning. Specifically, for a smoke image taken from one observation point, this application first determines any two smoke points in the smoke image taken from the observation point, and records them as the first smoke point and the second smoke point. For a smoke image taken from another observation point, this application determines the point corresponding to the position of the first smoke point and the point corresponding to the position of the second smoke point in the smoke image taken from the observation point, and records them as the third smoke point and the fourth smoke point. Furthermore, with the two observation point positions as endpoints, rays are drawn in the direction of the smoke points determined by their respective smoke images, so that each observation point can obtain two rays in different directions. This application uses rays pointing from two different observation points in the direction of any corresponding smoke point to determine an intersection in space (recorded as the first intersection), and uses rays pointing from two different observation points in the direction of another corresponding smoke point to obtain another intersection in space (recorded as the second intersection). It can be understood that two points can determine a straight line, and since the two points are in different positions in space, the straight line determined by the two points can intersect with the ground surface, and then this application can combine the straight line with the digital elevation model to determine the specific location of the fire. In this way, it avoids the situation where, when locating the fire based on the smoke in the initial stage, the smoke is at a higher position, resulting in no intersection with the digital elevation model, and thus the fire point cannot be located.

[0126] In one design, in order to ensure that the smoke points observed at two locations are consistent, the fire point locating method provided in the embodiment of the present application may further include:

[0127] S401: Determine a first spatial coordinate point of a first smoke point and a third spatial coordinate point of a third smoke point.

[0128] As a possible implementation, the electronic device can use a binocular vision algorithm to calculate the distance between the first image acquisition device and the first smoke point. Furthermore, the electronic device can use geometric principles to calculate the first spatial coordinate point of the first smoke point based on the spatial coordinates corresponding to the first image acquisition device, the pitch angle at which the first image acquisition device captures the first smoke image, and the distance between the first image acquisition device and the first smoke point. Similarly, the electronic device can use a binocular vision algorithm to calculate the distance between the second image acquisition device and the third smoke point. Furthermore, the electronic device can use geometric principles to calculate the third spatial coordinate point of the third smoke point based on the spatial coordinates corresponding to the second image acquisition device, the pitch angle at which the second image acquisition device captures the second smoke image, and the distance between the second image acquisition device and the third smoke point.

[0129] S402: Determine whether the deviation between the first space coordinate point and the third space coordinate point is less than or equal to a preset deviation.

[0130] As a possible implementation, the electronic device may determine a first sphere with a first spatial coordinate point as the center and a preset length as the radius, and determine a third sphere with a third spatial coordinate point as the center and the preset length as the radius. The electronic device uses the degree of overlap between the first and third spheres as the deviation between the first and third spatial coordinate points. Furthermore, the electronic device determines whether the degree of overlap is greater than or equal to a preset degree of overlap.

[0131] S403: When the deviation between the first spatial coordinate point and the third spatial coordinate point is less than or equal to the preset deviation, determine that the first smoke point and the third smoke point are the same spatial coordinate point.

[0132] As a possible implementation method, if the degree of overlap between the first sphere and the third sphere is greater than or equal to a preset overlap degree, which is equivalent to the deviation between the first spatial coordinate point and the third spatial coordinate point being less than or equal to a preset deviation, the electronic device determines that the first smoke point and the third smoke point are the same spatial coordinate point.

[0133] S404: When the deviation between the first spatial coordinate point and the third spatial coordinate point is greater than a preset deviation, determine that the first smoke point and the third smoke point are not the same spatial coordinate point.

[0134] As a possible implementation method, if the degree of overlap between the first sphere and the third sphere is less than a preset overlap degree, which is equivalent to the deviation between the first spatial coordinate point and the third spatial coordinate point being greater than a preset deviation, the electronic device determines that the first smoke point and the third smoke point are not the same spatial coordinate point.

[0135] For ease of understanding, the present embodiment exemplarily introduces a process for determining that the highest point of smoke observed at two points is the same point:

[0136] 1. Stereo correction.

[0137] The electronic device can perform stereoscopic correction on smoke images (such as the first smoke image and the second smoke image) taken at two different points at the same time. Figure 11 As shown in Figure 1, the purpose of stereo calibration is to mathematically project the left and right views of the same scene so that the two imaging planes are parallel to the baseline and the same point is located in the same row in the left and right images, which is referred to as coplanar row alignment.

[0138] 2. Stereo matching and disparity map calculation.

[0139] The purpose of stereo matching is to Figure 11 Each pixel in the left image is Figure 11 Find the corresponding point in the right image (i.e. the same physical point in the world), so that the disparity can be calculated: disparity = x i -x j , where x i and x j Represent the column coordinates of two corresponding points in the image.

[0140] The computational process of a stereo matching algorithm can include: matching cost calculation, cost aggregation, disparity optimization, and disparity refinement. Stereo matching algorithms can be divided into two categories: local methods, such as BM, SGM, ELAS, and Patch Match, and non-local methods, or global methods, such as Dynamic Programming, Graph Cut, and Belief Propagation.

[0141] 3. Calculate the distance (i.e. pixel depth).

[0142] After the electronic device obtains the disparity map, it can calculate the pixel depth. For example, using the StereoRectify() function in the cross-platform computer vision library (OpenCV) can obtain a reprojection matrix Q, which is a 4*4 disparity-to-depth mapping matrix: [X, Y, Z, W] T =Q*[x,y,disparity(x,y),1] T , use the Q matrix and cv2.reprojectImageTo3D to convert pixel coordinates into three-dimensional coordinates: This function returns a 3-channel matrix that stores the X, Y, and Z coordinates ( Figure 11 The left figure is in the camera coordinate system). Among them,

[0143]

[0144] c in the reprojection matrix Q x and c y is the coordinate of the left camera principal point in the image, f is the focal length, T x is the translation (negative value) of the projection center of the two cameras, also known as the baseline, which is equivalent to the translation vector T[0], c' x is the coordinate of the right camera's principal point in the image, and Z is the depth: Among them, f is the focal length (pixel focal length), b is the baseline length, d is the parallax, c is the distance between the pixels and the distance between the pixels. xr with c xl are the column coordinates of the two camera principal points.

[0145] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy to realize that the technical goals in this field are combined with the units and algorithm steps of each example described in the embodiments disclosed herein, and the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technical goals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0146] In an exemplary embodiment, the present application also provides a fire point positioning device. Figure 12 This is a schematic diagram of the composition of the fire point positioning device provided in the embodiment of the present application. Figure 12 As shown, the fire point locating device includes: an acquiring unit 501 and a determining unit 502 .

[0147] An acquisition unit 501 is configured to acquire a first smoke image and a second smoke image of a fire point at the same time; the first smoke image is acquired by a first image acquisition device located at a first observation position, and the second smoke image is acquired by a second image acquisition device located at a second observation position, where the first observation position is different from the second observation position; a determination unit 502 is configured to determine a first smoke point and a second smoke point in the first smoke image, and a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image; the determination unit 502 is further configured to determine a first pitch angle and a first horizontal angle of the first image acquisition device based on the first smoke point, and to determine a first ray with the first observation position of the first image acquisition device as the ray endpoint and the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle as the ray direction; the determination unit 502 is further configured to determine a second pitch angle and a second horizontal angle of the second image acquisition device based on the third smoke point, and to determine the second observation position of the second image acquisition device as the ray endpoint. , determining the second ray with the acquisition direction of the second image acquisition device at the second pitch angle and the second horizontal angle as the ray direction; the determining unit 502 is further used to determine the third pitch angle and the third horizontal angle of the first image acquisition device according to the second smoke point, and determine the third ray with the first observation position point of the first image acquisition device as the ray endpoint and the acquisition direction of the first image acquisition device at the third pitch angle and the third horizontal angle as the ray direction; the determining unit 502 is further used to determine the fourth pitch angle and the fourth horizontal angle of the second image acquisition device according to the fourth smoke point, and determine the fourth ray with the second observation position point of the second image acquisition device as the ray endpoint and the acquisition direction of the second image acquisition device at the fourth pitch angle and the fourth horizontal angle as the ray direction; the determining unit 502 is further used to determine a first intersection point according to the first ray and the second ray, and determine a second intersection point according to the third ray and the fourth ray; the determining unit 502 is further used to determine a target line according to the first intersection point and the second intersection point, and determine the location of the fire point according to the target line and the digital elevation model.

[0148] In one possible implementation, the determination unit 502 is specifically used to: obtain adjacent frame images of the first smoke image captured by the first image acquisition device; determine the changed area in the first smoke image based on the adjacent frame images and the first smoke image; determine the highest point of the changed area as the first smoke point; and determine the lowest point of the changed area as the second smoke point.

[0149] In a possible implementation, the determining unit 502 is specifically configured to determine an approximate intersection point between the first ray and the second ray, and use the approximate intersection point as the first intersection point.

[0150] In a possible implementation, the determining unit 502 is specifically configured to: determine a first target point on the first ray that is closest to the second ray, and determine a second target point on the second ray that is closest to the first ray; obtain a target line segment based on the first target point and the second target point; or determine any point on the target line segment as an approximate intersection point of the first ray and the second ray, or determine a point on the target line segment that satisfies a preset intercepting ratio as the approximate intersection point.

[0151] It should be noted that, Figure 12 The division of modules in the foregoing embodiments is merely illustrative, and is merely logical function division. In actual implementation, another division manner can be used. For example, two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of a software function unit.

[0152] In an example embodiment, the embodiment of the present application further provides a readable storage medium including software instructions, which, when running on an electronic device, cause the electronic device to execute any one of the methods provided by the above embodiments.

[0153] In an example embodiment, the embodiment of the present application further provides a computer program product including computer execution instructions, which, when running on an electronic device, cause the electronic device to execute any one of the methods provided by the above embodiments.

[0154] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer-executable instructions. When the computer-executable instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer-executable instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a storage medium (eg, a solid state disk (SSD)).

[0155] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple components. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0156] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely illustrative of the present application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, the present application is intended to include such modifications and variations as fall within the scope of the claims of the present application and their equivalents.

[0157] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any change or replacement within the technical scope disclosed by the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for locating a fire point, characterized in that: The method comprises: Acquire a first smoke image and a second smoke image of the fire point at the same time; the first smoke image is acquired by a first image acquisition device located at a first observation position, and the second smoke image is acquired by a second image acquisition device located at a second observation position, where the first observation position and the second observation position are different; determining a first smoke point and a second smoke point in the first smoke image, and determining a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image; Determine a first pitch angle and a first horizontal angle of the first image acquisition device based on the first smoke point, and determine a first ray with the first observation position of the first image acquisition device as the ray endpoint and the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle as the ray direction; determining a second pitch angle and a second horizontal angle of the second image acquisition device based on the third smoke point, and determining a second ray with the second observation position of the second image acquisition device as the ray endpoint and the acquisition direction of the second image acquisition device at the second pitch angle and the second horizontal angle as the ray direction; Determine a third pitch angle and a third horizontal angle of the first image acquisition device based on the second smoke point, and determine a third ray with the first observation position of the first image acquisition device as the ray endpoint and the acquisition direction of the first image acquisition device at the third pitch angle and the third horizontal angle as the ray direction; determining a fourth pitch angle and a fourth horizontal angle of the second image acquisition device based on the fourth smoke point, and determining a fourth ray with the second observation position of the second image acquisition device as an endpoint of the ray and a capture direction of the second image acquisition device at the fourth pitch angle and the fourth horizontal angle as a ray direction; Determine a first intersection point based on the first ray and the second ray, and determine a second intersection point based on the third ray and the fourth ray; A target straight line is determined according to the first intersection point and the second intersection point, and a fire point position is determined according to the target straight line and a digital elevation model.

2. The method according to claim 1, characterized in that The determining of the first smoke point and the second smoke point in the first smoke image includes: Acquiring adjacent frame images of the first smoke image captured by the first image acquisition device; determining a changed region in the first smoke image according to the adjacent frame images and the first smoke image; determining the highest point of the change area as the first smoke point; The lowest point of the change area is determined as the second smoke point.

3. The method according to claim 1, characterized in that The determining of a first intersection point according to the first ray and the second ray includes: An approximate intersection point between the first ray and the second ray is determined, and the approximate intersection point is used as a first intersection point.

4. The method according to claim 3, characterized in that The determining an approximate intersection point of the first ray and the second ray includes: Determine a first target point on the first ray that is closest to the second ray, and determine a second target point on the second ray that is closest to the first ray; Obtaining a target line segment based on the first target point and the second target point; Any point on the target line segment is used as the approximate intersection point of the first ray and the second ray, or a point on the target line segment that meets a preset interception ratio is determined as the approximate intersection point.

5. A method for locating a fire point, characterized in that: The method comprises: Acquire a first smoke image and a second smoke image of the fire point at the same time; the first smoke image is acquired by a first image acquisition device located at a first observation position, and the second smoke image is acquired by a second image acquisition device located at a second observation position, where the first observation position and the second observation position are different; determining a first smoke point and a second smoke point in the first smoke image, and determining a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image; Determine a first spatial coordinate point of the first smoke point, a second spatial coordinate point of the second smoke point, a third spatial coordinate point of the third smoke point, and a fourth spatial coordinate point of the fourth smoke point; If the deviation between the first spatial coordinate point and the third spatial coordinate point is less than or equal to a preset deviation, determining a first target point based on the first smoke point and the third smoke point; and if the deviation between the second spatial coordinate point and the fourth spatial coordinate point is less than or equal to the preset deviation, determining a second target point based on the third smoke point and the fourth smoke point; A target line is determined based on the first target point and the second target point, and a fire point location is determined based on the target line and a digital elevation model.

6. The method according to claim 5, characterized in that The determining of the first target point according to the first smoke point and the third smoke point includes: determining the first smoke point as the first target point; or, determining the third smoke point as the first target point; or, determining the midpoint between the first smoke point and the third smoke point as the first target point; or, According to a preset calculation ratio, a point on the line connecting the first smoke point and the third smoke point is determined as the first target point.

7. The method according to any one of claims 1 to 6, characterized in that Determining the location of the fire point according to the target straight line and the digital elevation model includes: The intersection of the target straight line and the digital elevation model is determined as the fire starting point.

8. A fire point locating device, characterized in that: The device includes an acquisition unit and a determination unit; The acquisition unit is configured to acquire a first smoke image and a second smoke image of the fire point at the same time; the first smoke image is acquired by a first image acquisition device located at a first observation position, and the second smoke image is acquired by a second image acquisition device located at a second observation position, the first observation position and the second observation position being different; The determining unit is configured to determine a first smoke point and a second smoke point in the first smoke image, and to determine a third smoke point corresponding to the first smoke point and a fourth smoke point corresponding to the second smoke point in the second smoke image; The determining unit is further configured to determine a first pitch angle and a first horizontal angle of the first image acquisition device based on the first smoke point, and determine a first ray with the first observation position of the first image acquisition device as the ray endpoint and the acquisition direction of the first image acquisition device at the first pitch angle and the first horizontal angle as the ray direction; The determining unit is further configured to determine a second pitch angle and a second horizontal angle of the second image acquisition device based on the third smoke point, and determine a second ray using the second observation position of the second image acquisition device as a ray endpoint and a capture direction of the second image acquisition device at the second pitch angle and the second horizontal angle as a ray direction; The determining unit is further configured to determine a third pitch angle and a third horizontal angle of the first image acquisition device based on the second smoke point, and determine a third ray using the first observation position of the first image acquisition device as a ray endpoint and a capture direction of the first image acquisition device at the third pitch angle and the third horizontal angle as a ray direction; The determining unit is further configured to determine a fourth pitch angle and a fourth horizontal angle of the second image acquisition device based on the fourth smoke point, and determine a fourth ray using the second observation position of the second image acquisition device as a ray endpoint and a capture direction of the second image acquisition device at the fourth pitch angle and the fourth horizontal angle as a ray direction; The determining unit is further configured to determine a first intersection point based on the first ray and the second ray, and determine a second intersection point based on the third ray and the fourth ray; The determining unit is further configured to determine a target straight line based on the first intersection point and the second intersection point, and to determine a fire point location based on the target straight line and a digital elevation model.

9. An electronic device, characterized in that: include: processor and memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 7.

10. A fire point location system, characterized in that: The device comprises the electronic device as claimed in claim 9 and a plurality of image acquisition devices, wherein the plurality of image acquisition devices are used to acquire smoke images of the fire point at different observation positions and transmit the acquired smoke images to the electronic device.

11. A readable storage medium, characterized in that: The readable storage medium includes: software instructions; When the software instructions are executed in an electronic device, the electronic device is enabled to implement the method according to any one of claims 1 to 7.

Images