A fire point positioning method and device, electronic equipment and storage medium

Abstract

This invention relates to a fire point location method, apparatus, electronic device, and storage medium, belonging to the field of location technology. The invention utilizes a first detection image to obtain focal length parameters, and focuses on the flame target based on these parameters, accurately obtaining the target area containing the flame target. A thermal infrared emitter is used to identify fire points within the target area, and infrared ranging is performed on these fire points, thereby improving the accuracy and efficiency of fire point location and effectively reducing manual intervention in the fire point location process.

Description

Technical Field

[0001] This invention relates to the field of positioning technology, and in particular to a fire point positioning method, device, electronic device, and storage medium. Background Technology

[0002] Existing fire location methods mainly determine the location of the fire point by detecting fixed heat sources. However, this method is difficult to achieve precise fire point location. It is necessary to transmit the detected heat source information to the backend, and then have a person estimate the location and distance of the fire point by visual inspection in order to achieve reliable fire point location and facilitate subsequent feedback responses such as fire extinguishing.

[0003] Based on the above technical analysis, current fire point location methods suffer from drawbacks such as low location efficiency, low location accuracy, and high susceptibility to human intervention. Summary of the Invention

[0004] To address the shortcomings of existing fire point location methods, such as low location efficiency, low location accuracy, and high susceptibility to human intervention, this invention provides a fire point location method, device, electronic device, and storage medium.

[0005] In a first aspect, to solve the above-mentioned technical problems, the present invention provides a fire point location method, comprising:

[0006] The first detection image is obtained with the location of the flame target as the image center.

[0007] Based on the first detected image, determine the focal length parameters corresponding to the flame target;

[0008] Based on the focal length parameter, determine the target area where the flame target is located;

[0009] The location of the fire point in the target area is determined by using a thermal infrared emitter.

[0010] The beneficial effects of the present invention are: by using the first detection image to obtain the focal length parameter, and focusing on the flame target based on the focal length parameter, the target area containing the flame target can be accurately obtained. The fire point in the target area is identified by the thermal infrared emitter, and the distance of the fire point is measured, thereby improving the accuracy and efficiency of fire point positioning, and effectively reducing manual intervention in the fire point positioning process.

[0011] Furthermore, the above-mentioned determination of the focal length parameters corresponding to the flame target based on the first detected image includes:

[0012] Obtain the image size of the first detected image;

[0013] Based on the pre-trained flame target detection model and the first detection image, the anchor box size of the target anchor box corresponding to the flame target is determined;

[0014] Based on the anchor frame size and the image size of the first detection image, determine the size proportion of the flame target in the first detection image;

[0015] Based on the size ratio, the focal length parameters corresponding to the flame target are determined.

[0016] The beneficial effect of adopting the above-mentioned improvement scheme is that, by measuring the size proportion of the flame target in the first detection image, the focal length parameters applicable to the flame target can be reasonably determined.

[0017] Furthermore, the aforementioned flame target detection model was trained in the following way:

[0018] Collect videos corresponding to different scenarios;

[0019] Images containing flame targets are collected from each video to obtain a valid dataset;

[0020] Data augmentation is performed on the valid dataset to obtain an expanded dataset;

[0021] The initial flame target detection model is iteratively trained based on the expanded dataset until the loss function value of the initial flame target detection model meets the preset training termination condition. The initial flame target detection model at the end of training is then determined as the flame target detection model. The model structure of the initial flame target detection model is the Yolov5x model structure.

[0022] The beneficial effects of adopting the above-mentioned improvement scheme are: to establish a dataset for model training using videos from different scenarios, and to perform data augmentation on the dataset, thereby effectively improving the wide applicability of the flame target detection model.

[0023] Furthermore, the aforementioned method of using a thermal infrared emitter to determine the location of a fire point in a target area includes:

[0024] Thermal images of the target area are acquired using a thermal infrared emitter, where the deflection angle of the thermal infrared emitter varies for pixels at different locations in the thermal image.

[0025] Obtain target points in a thermal image that meet set conditions, wherein the set conditions are that the gray value of a pixel is equal to the gray value of the target pixel in the thermal image, and the gray value of each pixel in the set surrounding area corresponding to the target pixel is greater than the set gray value, wherein the target pixel is the pixel corresponding to the maximum gray value among all pixels in the thermal image.

[0026] Multiple laser ranging measurements were performed on the target point to obtain multiple target distances between the target point and the thermal infrared emitter.

[0027] Based on the distances to each target, the location of the fire point in the target area is determined.

[0028] The beneficial effects of adopting the above-mentioned improvement scheme are: by using the gray values ​​in the thermal image, the fire point in the target area can be quickly found, and by measuring the distance to the fire point multiple times, the accuracy of fire point location can be further improved.

[0029] Furthermore, the above-mentioned determination of the fire location within the target area based on the distances to each target includes:

[0030] Determine the average distance to each target and use the average as the fire point distance;

[0031] Obtain the target deflection angle of the thermal infrared emitter corresponding to the target point;

[0032] Based on the pre-established self-coordinate system, the distance to the fire point, and the target deflection angle, determine the first coordinate of the fire point in its own coordinate system;

[0033] The first coordinate is converted to the second coordinate in the world coordinate system, and the latitude, longitude and altitude coordinates corresponding to the second coordinate are obtained to obtain the fire point location in the target area;

[0034] The target deflection angle includes the horizontal rotation angle, the side plane rotation angle, and the vertical rotation angle. The origin of its own coordinate system is the position of the thermal infrared emitter, and the first coordinate is determined by the following formula:

[0035] x h =0+l*sinα y *cosα z

[0036] y h =0+l*sinα x *cosα z

[0037] z h =0+l*sinα x *cosα y

[0038] Where, x h y h and z h The first coordinate represents the coordinates of the first coordinate along the x, y, and z axes of its own coordinate system, l represents the distance to the fire point, and α represents the distance to the fire point. x α represents the angle of rotation in the horizontal plane. y α represents the included angle of rotation of the lateral plane. z Indicates the included angle of vertical rotation.

[0039] The beneficial effects of adopting the above-mentioned improved scheme are: by using the target deflection angle and the distance to the fire point, the first coordinate of the fire point in its own coordinate system can be quickly calculated, and then based on the first coordinate, the latitude, longitude and altitude information of the fire point can be determined by coordinate transformation, thereby completing the location of the fire point.

[0040] Furthermore, the above-mentioned determination of the target area where the flame target is located based on the focal length parameter includes:

[0041] Based on the focal length parameter, a set number of video frames are randomly acquired from the flame target within a set time period to obtain multiple target video frames.

[0042] For each target video frame, based on the trained flame target detection model, it is detected whether the target video frame is a flame image, where a flame image is an image containing a flame target;

[0043] If the number of detected flame images is greater than the set value, the flame target detection model and flame images are used to determine the target box corresponding to the flame target in the flame image, and the area corresponding to the target box is determined as the target area where the flame target is located.

[0044] If the number of detected flame images is not greater than the set value, images from different regions are acquired until the first detected image with the flame target's location as the image center is acquired again.

[0045] The beneficial effect of adopting the above-mentioned improvement scheme is that by performing multiple flame target recognitions on video frames, the false recognition rate of flame targets can be reduced, ensuring the effectiveness of the subsequently acquired fire point locations.

[0046] Furthermore, the acquisition of the first detection image with the location of the flame target as the image center position includes:

[0047] Acquire a second detection image containing the flame target;

[0048] Obtain the center point position of the flame target in the second detection image to obtain the first center point position;

[0049] Obtain the center point position of the second detection image to get the second center point position;

[0050] Check whether the positions of the first center point and the second center point are the same;

[0051] If the position of the first center point is the same as the position of the second center point, then the second detection image is determined as the first detection image;

[0052] If the position of the first center point is different from the position of the second center point, the first shooting angle corresponding to the second detection image is obtained. Based on the distance information between the positions of the first and second center points, the rotation angle of the acquisition device is determined. Based on the first shooting angle and the rotation angle, the second shooting angle corresponding to the first detection image is determined. According to the second shooting angle, the flame target is image acquired to obtain the first detection image.

[0053] The beneficial effects of adopting the above-mentioned improved scheme are: by acquiring the first detection image with the flame target as the image center, the integrity of the flame target contained in the first detection image can be guaranteed, and it is beneficial to the subsequent location of the fire point.

[0054] In a second aspect, the present invention provides a fire point locating device, comprising:

[0055] The acquisition module is used to acquire the first detection image with the position of the flame target as the image center position;

[0056] The first processing module is used to determine the focal length parameters corresponding to the flame target based on the first detected image;

[0057] The second processing module is used to determine the target area where the flame target is located based on the focal length parameter;

[0058] The positioning module is used to determine the location of fire points in the target area using a thermal infrared emitter.

[0059] Thirdly, the present invention provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform all or part of the steps of the fire point location method of the first aspect.

[0060] Fourthly, the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement all or part of the steps of the fire point location method of the first aspect. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating a fire point location method provided in an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of a fire point locating device provided in an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0064] The following embodiments are further explanations and supplements to the present invention and do not constitute any limitation on the present invention.

[0065] The following describes a fire point location method according to an embodiment of the present invention with reference to the accompanying drawings.

[0066] Reference Figure 1 As shown, the present invention provides a fire point location method, comprising the following steps S1 to S4, wherein:

[0067] In step S1, a first detection image is acquired with the position of the flame target as the image center position.

[0068] In this embodiment, the first detected image is a visible light image, which can be acquired by a camera mounted on the drone.

[0069] Optionally, in one embodiment, the process of acquiring the first detection image with the location of the flame target as the image center includes:

[0070] Acquire a second detection image containing the flame target;

[0071] Obtain the center point position of the flame target in the second detection image to obtain the first center point position;

[0072] Obtain the center point position of the second detection image to get the second center point position;

[0073] Check whether the positions of the first center point and the second center point are the same;

[0074] If the position of the first center point is the same as the position of the second center point, then the second detection image is determined as the first detection image;

[0075] If the position of the first center point is different from the position of the second center point, the first shooting angle corresponding to the second detection image is obtained. Based on the distance information between the positions of the first and second center points, the rotation angle of the acquisition device is determined. Based on the first shooting angle and the rotation angle, the second shooting angle corresponding to the first detection image is determined. According to the second shooting angle, the flame target is image acquired to obtain the first detection image.

[0076] For example, a coordinate system containing a vertical Z-axis and a horizontal X-axis is set with the UAV as the origin, enabling the UAV to acquire detection images in real time during flight and perform flame target detection on the acquired images. When a flame target is detected (i.e., the second detection image is acquired), the pixel interval distance (P) between the first center point position and the second center point position in the vertical direction (Z-axis direction) and the horizontal direction (X-axis direction) is obtained. z P xThe system obtains the rotation angles (α) of the drone's camera relative to the Z-axis and X-axis when acquiring the second detection image. z1 α x1 The rotation angle (α) z1 α x1 That is, the first shooting angle corresponding to the second detection image.

[0077] Based on the linear relationship coefficients (k) generated from previous statistics z k x The first shooting angle and the rotation angle are superimposed to obtain the second shooting angle. Keeping other parameters of the drone's camera unchanged, the camera is rotated to this second shooting angle to acquire the detection image, thus obtaining the first detection image with the position of the flame target as the image center.

[0078] As one possible implementation method, the first shooting angle is determined by the following formula:

[0079]

[0080] Among them, (α) z1 α x1 (k) represents the first shooting angle, (k) z k x ) represents the coefficient of the linear relationship in the mapping, (P) z P x ) represents the pixel spacing distance between the first center point position and the second center point position.

[0081] In step S2, the focal length parameters corresponding to the flame target are determined based on the first detected image.

[0082] It is understandable that the focal length parameter corresponding to the flame target is used to improve the imaging effect of the flame target. For example, by focusing and shooting the flame target based on the focal length parameter, a clear image of the flame target can be obtained, thereby improving the recognition accuracy of the flame target.

[0083] Optionally, in one embodiment, the process of determining the focal length parameter corresponding to the flame target based on the first detected image includes:

[0084] Obtain the image size of the first detected image;

[0085] Based on the pre-trained flame target detection model and the first detection image, the anchor box size of the target anchor box corresponding to the flame target is determined;

[0086] Based on the anchor frame size and the image size of the first detection image, determine the size proportion of the flame target in the first detection image;

[0087] Based on the size ratio, the focal length parameters corresponding to the flame target are determined.

[0088] For example, the focal length parameter corresponding to the flame target is determined by the following inequality:

[0089]

[0090] Where P represents the size percentage of the flame target in the first detection image, and 2x zoom, 4x zoom, and 6x zoom represent the type of focal length parameter.

[0091] Optionally, in one embodiment, the above-described flame target detection model is trained in the following manner:

[0092] Collect videos corresponding to different scenarios;

[0093] Images containing flame targets are collected from each video to obtain a valid dataset;

[0094] Data augmentation is performed on the valid dataset to obtain an expanded dataset;

[0095] The initial flame target detection model is iteratively trained based on the expanded dataset until the loss function value of the initial flame target detection model meets the preset training termination condition. The initial flame target detection model at the end of training is then determined as the flame target detection model. The model structure of the initial flame target detection model is the Yolov5x model structure.

[0096] In this embodiment, an initial flame target detection model is established using deep learning technology and the Yolov5x model structure. The model is then trained and used to detect flame targets, thereby enabling the identification of flame targets in visible light images captured by the camera.

[0097] For example, more than fifty hours of flame video are collected from multiple scenes, ensuring that each video includes time periods such as noon, night, dusk, and dawn. Each flame video can include scenes such as forests, grasslands, highways, squares, and residential areas, and each video contains target acquisition distances ranging from 10m to 5000m, where the target acquisition distance represents the distance between the flame target and the acquisition device. The time, location, and target acquisition distance in each flame video should be approximately evenly distributed to ensure data source diversity and balance. The same set number of flame images are selected from each flame video to obtain a valid dataset of 200,000 flame images.

[0098] To improve the fit of the later model and make it more adaptable to the color calibration of cameras from different brands, while also being able to cope with the loss of real data from cameras to a certain extent, scene augmentation processing (data augmentation processing) can be performed on the effective dataset.

[0099] The scene enhancement processing includes Gaussian blur, mean blur, maximum or minimum value blur, median blur, bilateral blur, noise interference, fog effect, raindrop effect, grayscale conversion, color transformation, slight random distortion, and brightness adjustment. This expands the original 200,000 flame images proportionally to over 800,000, resulting in an expanded dataset. This enriches the data diversity and improves the model's broad applicability.

[0100] The expanded dataset can be divided into training and testing sets in an 8:2 ratio. The training set is input into the initial flame target detection model for training. After a training process with a batch size of 256 and more than 15,000 epochs (the number of times training is performed using all samples in the training set), the model converges (i.e., the loss function value meets the preset training termination condition), and the model training is complete. After testing on the test set, the model performs well and can be directly used for flame target detection. The model test results are shown below:

[0101] accuracy Accuracy Recall rate 94.9％ 96.3％ 95.6％

[0102] As one possible implementation, the flame type corresponding to the flame target is determined based on the size proportion of the flame target in the first detection image. This flame type is determined by the following inequality:

[0103]

[0104] Where P represents the size percentage of the flame target in the first detection image, and large fire, medium fire, small fire, and sparks represent the flame type.

[0105] In step S3, the target area where the flame target is located is determined based on the focal length parameter.

[0106] In this embodiment, by using the obtained focal length parameters and keeping the camera position unchanged, a better image is obtained with the flame target as the image center, and the target area where the flame target is located is determined based on the obtained image.

[0107] Optionally, in one embodiment, the process of determining the target area where the flame target is located based on the focal length parameter includes:

[0108] Based on the focal length parameter, a set number of video frames are randomly acquired from the flame target within a set time period to obtain multiple target video frames.

[0109] For each target video frame, based on the trained flame target detection model, it is detected whether the target video frame is a flame image, where a flame image is an image containing a flame target;

[0110] If the number of detected flame images is greater than the set value, the flame target detection model and flame images are used to determine the target box corresponding to the flame target in the flame image, and the area corresponding to the target box is determined as the target area where the flame target is located.

[0111] If the number of detected flame images is not greater than the set value, images from different regions are acquired until the first detected image with the flame target's location as the image center is acquired again.

[0112] For example, keeping the camera position unchanged, the obtained focal length parameters are used to continuously capture video of the flame target for 2 seconds, and 30 frames are randomly selected from the captured video as target video frames. Flame target detection is performed on these 30 frames. If the flame target is successfully identified in more than 80% of the target video frames, the target bounding box corresponding to the flame target in any target video frame is acquired; otherwise, the normal scanning mode is returned. Even if the drone continues to move and acquire images in real time for flame target detection, the drone stops moving and begins acquiring the first detection image when the flame target is detected again in the acquired images.

[0113] In step S4, the location of the fire point in the target area is determined using a thermal infrared emitter.

[0114] It is understandable that the fire point and its surrounding area are high-temperature areas, and the fire point is the location with the highest temperature in this high-temperature area. Therefore, the temperature characteristics of each location in the target area can be obtained through thermal sensing and other methods, thereby finding the fire point.

[0115] In this embodiment, a thermal infrared emitter emits multiple thermal infrared rays toward the target area to detect high-temperature areas in the target area and determine the distance between the fire point in the high-temperature area and the thermal infrared emitter, thereby determining the spatial location of the fire point.

[0116] Optionally, in one embodiment, the process of determining the location of a fire point in a target area using a thermal infrared emitter includes:

[0117] Thermal images of the target area are acquired using a thermal infrared emitter, where the deflection angle of the thermal infrared emitter varies for pixels at different locations in the thermal image.

[0118] Obtain target points in a thermal image that meet set conditions, wherein the set conditions are that the gray value of a pixel is equal to the gray value of the target pixel in the thermal image, and the gray value of each pixel in the set surrounding area corresponding to the target pixel is greater than the set gray value, wherein the target pixel is the pixel corresponding to the maximum gray value among all pixels in the thermal image.

[0119] Multiple laser ranging measurements were performed on the target point to obtain multiple target distances between the target point and the thermal infrared emitter.

[0120] Based on the distances to each target, the location of the fire point in the target area is determined.

[0121] For example, an infrared emitter on a drone emits infrared light and simultaneously receives visible absolute red light. Linear detection is performed on the absolute red light to detect its endpoint. If the endpoint is not within the target area, the deflection angle of the infrared emitter is adjusted based on the endpoint, the center point of the target area, and the current angle of the emitted infrared light (i.e., the current deflection angle of the infrared emitter) to ensure the endpoint of the infrared light falls within the target area. The infrared emitter scans each point within the target area laterally or vertically with the minimum deflection angle and performs boundary detection. If a boundary is exceeded (i.e., the endpoint of the infrared light is outside the target area), the process jumps directly to the next row / column within the target area. Otherwise, the grayscale value of the point and the corresponding deflection angle of the infrared emitter are acquired and stored in memory. This deflection angle includes the deflection angle (α) of the infrared emitter relative to the X, Y, and Z axes in its own coordinate system. x α y α z ).

[0122] The system identifies the target pixel with the highest grayscale value in the thermal image, which corresponds to the location with the highest temperature in the target area. If the grayscale values ​​of all pixels in the surrounding area of ​​the target pixel (e.g., a 4x4 pixel area) are greater than 250, the target pixel is identified as the target point. A high-sensitivity laser rangefinder is used to measure the target distance between the target point and the thermal infrared emitter, obtaining the target distance l = v (the speed of infrared light) * T (the round-trip time of infrared light) / 2. Three consecutive laser rangefinder measurements are performed on the target point, and the average of the obtained target distances is taken as the fire point distance. If no target point meeting the set conditions can be found in the thermal image, it is determined that there is no fire point in the target area, and the system returns to the environmental scanning state, allowing the UAV to continue flying and detecting flame targets in real time.

[0123] Optionally, in one embodiment, determining the location of the fire point in the target area based on each target distance includes:

[0124] Determine the average distance to each target and use the average as the fire point distance;

[0125] Obtain the target deflection angle of the thermal infrared emitter corresponding to the target point;

[0126] Based on the pre-established self-coordinate system, the distance to the fire point, and the target deflection angle, determine the first coordinate of the fire point in its own coordinate system;

[0127] The first coordinate is converted to the second coordinate in the world coordinate system, and the latitude, longitude and altitude coordinates corresponding to the second coordinate are obtained to obtain the fire point location in the target area;

[0128] The target deflection angle includes the horizontal rotation angle, the side plane rotation angle, and the vertical rotation angle. The origin of its own coordinate system is the position of the thermal infrared emitter, and the first coordinate is determined by the following formula:

[0129] x h =0+l*sinα y *cosα z

[0130] y h =0+l*sinα x *cosα z

[0131] z h =0+l*sinα x *cosα y

[0132] Where, x h y h and z h The first coordinate represents the coordinates of the first coordinate along the x, y, and z axes of its own coordinate system, l represents the distance to the fire point, and α represents the distance to the fire point. x α represents the angle of rotation in the horizontal plane. y α represents the included angle of rotation of the lateral plane. z Indicates the included angle of vertical rotation.

[0133] It should be noted that the coordinate transformation formula in computer graphics, which converts a local coordinate system to a world coordinate system, can be used to transform the first coordinate in the local coordinate system to the second coordinate in the world coordinate system. The coordinate transformation formula is expressed as follows:

[0134]

[0135] Among them, v Localv represents the coordinates in the local coordinate system (its own coordinate system). Global Represents the coordinates in the world coordinate system, R represents the rotation matrix, T represents the translation matrix, (p x p y p z (r) represents the origin of the local coordinate system. x r y r z ) represents the unit normal vector of the r-axis in the local coordinate system.

[0136] That is, for the same object, its coordinates in the world coordinate system are equal to its coordinates at the origin of the local coordinate system plus the unit normal vector of the r-axis of the local coordinate system (which can be understood as the x-axis of the local coordinate system).

[0137] Finally, using the conversion formulas of the national 80 and WGS84 coordinate systems, the second coordinate (x, y, z) is converted into latitude and longitude data (N, W) and altitude coordinate data H, thus obtaining the fire point coordinates (N, W, H).

[0138] The fire point location method provided in the above embodiments, by combining GIS (Geographic Information System) location information, CV (Computer Vision) target detection technology, sensor technology and coordinate system transformation algorithm, realizes the function of precise 3D fire point location, and provides technical support for subsequent intelligent operation.

[0139] By combining target detection technology from CV (Computer Vision) with a heat source sensor (thermal infrared emitter) and video frames, the system achieves the function of detecting and judging flame targets, greatly improving recognition accuracy and reducing false alarms. After identifying the fire point, the system uses distance positioning via the heat source sensor to calculate the distance to the device currently emitting infrared light. Combined with GIS map information, the precise 3D location of the fire point is calculated. The drone can then travel to the designated target location to accurately extinguish the fire. During flight, the drone can automatically sample the 3D location of the fire point in real time, enabling route updates and corrections. This solves the problem that the inability of the drone to accurately locate the fire point leads to the inaccurate deployment of fire extinguishing equipment, resulting in fire extinguishing failure and greatly increasing the possibility of uncontrolled fire spread.

[0140] like Figure 2 As shown, an embodiment of the present invention provides a fire point locating device, comprising:

[0141] The acquisition module 20 is used to acquire a first detection image with the position of the flame target as the image center position;

[0142] The first processing module 30 is used to determine the focal length parameters corresponding to the flame target based on the first detected image;

[0143] The second processing module 40 is used to determine the target area where the flame target is located based on the focal length parameter;

[0144] The positioning module 50 is used to determine the location of the fire point in the target area using a thermal infrared emitter.

[0145] Optionally, the first processing module 30 is specifically used to obtain the image size of the first detection image; determine the anchor frame size of the target anchor frame corresponding to the flame target based on the pre-trained flame target detection model and the first detection image; determine the size proportion of the flame target in the first detection image based on the anchor frame size and the image size of the first detection image; and determine the focal length parameter corresponding to the flame target based on the size proportion.

[0146] Optionally, the first processing module 30 is also used to collect videos corresponding to different scenes; collect images containing flame targets from each video to obtain a valid dataset; perform data augmentation processing on the valid dataset to obtain an expanded dataset; iteratively train the initial flame target detection model based on the expanded dataset until the loss function value of the initial flame target detection model meets the preset training termination condition, and determine the initial flame target detection model at the end of training as the flame target detection model, wherein the model structure of the initial flame target detection model is the Yolov5x model structure.

[0147] Optionally, the positioning module 50 is specifically used to acquire a thermally sensed image of the target area using a thermally sensed infrared emitter, wherein the deflection angle of the thermally sensed infrared emitter corresponds to different pixels at different positions in the thermally sensed image; acquire target points in the thermally sensed image that meet set conditions, wherein the set conditions are that the gray value of the pixel is equal to the gray value of the target pixel in the thermally sensed image, and the gray value of each pixel in the set surrounding area corresponding to the target pixel is greater than the set gray value, wherein the target pixel is the pixel corresponding to the maximum gray value among all pixels in the thermally sensed image; perform multiple laser ranging on the target point to obtain multiple target distances between the target point and the thermally sensed infrared emitter; and determine the fire point position in the target area based on each target distance.

[0148] Optionally, the positioning module 50 is also used to determine the average distance of each target, using the average as the fire point distance; obtain the target deflection angle of the thermal infrared emitter corresponding to the target point; determine the first coordinate of the fire point in its own coordinate system based on the pre-established self-coordinate system, the fire point distance, and the target deflection angle; convert the first coordinate to a second coordinate in the world coordinate system, and obtain the latitude, longitude, and altitude coordinate data corresponding to the second coordinate to obtain the fire point position in the target area; wherein, the target deflection angle includes the horizontal plane rotation angle, the side plane rotation angle, and the vertical rotation angle, the origin of the self-coordinate system is the position of the thermal infrared emitter, and the first coordinate is determined by the following formula:

[0149] x h =0+l*sinα y *cosα z

[0150] y h =0+l*sinα x *cosα z

[0151] z h =0+l*sinα x *cosα y

[0152] Where, x h y h and z h The first coordinate represents the coordinates of the first coordinate along the x, y, and z axes of its own coordinate system, l represents the distance to the fire point, and α represents the distance to the fire point. x α represents the angle of rotation in the horizontal plane. y α represents the included angle of rotation of the lateral plane. z Indicates the included angle of vertical rotation.

[0153] Optionally, the second processing module 40 is specifically used to randomly acquire a set number of video frames of the flame target within a set time period based on the focal length parameter, thereby obtaining multiple target video frames; for each target video frame, based on the trained flame target detection model, detect whether the target video frame is a flame image, wherein a flame image is an image containing a flame target; if the number of detected flame images is greater than a set value, then the flame target detection model and the flame image are used to determine the target box corresponding to the flame target in the flame image, and the area corresponding to the target box is determined as the target area where the flame target is located; if the number of detected flame images is not greater than a set value, then the acquired images of different areas are obtained until the first detection image with the position of the flame target as the image center position is re-acquired.

[0154] Optionally, the acquisition module 20 is specifically used to acquire a second detection image containing a flame target; acquire the center point position of the flame target in the second detection image to obtain a first center point position; acquire the center point position of the second detection image to obtain a second center point position; detect whether the first center point position and the second center point position are the same; if the first center point position and the second center point position are the same, then the second detection image is determined as the first detection image; if the first center point position and the second center point position are not the same, then the first shooting angle corresponding to the second detection image is acquired, the rotation angle of the acquisition device is determined based on the distance information between the first center point position and the second center point position, and the second shooting angle corresponding to the first detection image is determined based on the first shooting angle and the rotation angle, and the flame target is image acquired according to the second shooting angle to obtain the first detection image.

[0155] The present invention provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the steps of the fire point location method of any of the above embodiments.

[0156] like Figure 3 As shown, an electronic device 500 provided in this embodiment of the invention includes a memory 510, a processor 520, and a program 530 stored in the memory 510 and running on the processor 520. When the processor 520 executes the program 530, it implements the steps of the fire point location method of any of the above embodiments.

[0157] Among them, the electronic device 500 can be a computer, mobile phone, etc., and correspondingly, its program 530 is computer software or mobile phone App, etc. The parameters and steps of the electronic device 500 of the present invention can be referred to the parameters and steps in the embodiment of the fire point location method above, and will not be repeated here.

[0158] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this disclosure can be embodied in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0159] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0160] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for locating a fire point, characterized in that, include: Acquire the first detection image with the location of the flame target as the image center, including: Acquire a second detection image containing the flame target; The center point position of the flame target in the second detected image is obtained to obtain the first center point position; Obtain the center point position of the second detected image to obtain the second center point position; Detect whether the positions of the first center point and the second center point are the same; If the position of the first center point is the same as the position of the second center point, then the second detected image is determined as the first detected image; If the position of the first center point is different from the position of the second center point, the first shooting angle corresponding to the second detection image is obtained. Based on the distance information between the position of the first center point and the position of the second center point, the rotation angle of the acquisition device is determined. Based on the first shooting angle and the rotation angle, the second shooting angle corresponding to the first detection image is determined. According to the second shooting angle, the flame target is image acquired to obtain the first detection image. Based on the first detected image, determine the focal length parameters corresponding to the flame target; Based on the focal length parameter, the target area where the flame target is located is determined; Determining the location of a fire point in the target area using a thermal infrared emitter includes: A thermal image of the target area is acquired using a thermal infrared emitter, wherein the deflection angle of the thermal infrared emitter corresponds to different pixels at different locations in the thermal image. Obtain target points in the thermal image that meet set conditions, wherein the set conditions are that the gray value of a pixel is equal to the gray value of the target pixel in the thermal image, and the gray value of each pixel in the set surrounding area corresponding to the target pixel is greater than the set gray value, wherein the target pixel is the pixel corresponding to the largest gray value among all pixels in the thermal image. Multiple laser ranging measurements are performed on the target point to obtain multiple target distances between the target point and the thermal infrared emitter. Based on the respective target distances, the location of the fire point in the target area is determined, including: Determine the average value of each of the target distances, and use the average value as the fire point distance; Obtain the target deflection angle of the thermal infrared emitter corresponding to the target point; Based on the pre-established self-coordinate system, the fire point distance, and the target deflection angle, the first coordinate of the fire point in the self-coordinate system is determined; The first coordinate is converted into a second coordinate in the world coordinate system, and the latitude, longitude and altitude coordinates corresponding to the second coordinate are obtained to obtain the fire point location in the target area; The target deflection angle includes the horizontal plane rotation angle, the side plane rotation angle, and the vertical rotation angle. The origin of the self-coordinate system is the position of the thermal infrared emitter. The first coordinate is determined by the following formula: x h = 0 + l*sinα y *cosα z y h = 0 + l*sinα x *cosα z z h = 0 + l*sin a x *cos a y wherein x h , y h and z h represent the coordinate values of the first coordinate in the x, y and z axes of the self-coordinate system, l represents the fire point distance, a x represents the horizontal plane rotation angle, a y represents the side plane rotation angle, and a z represents the vertical rotation angle.

2. The method according to claim 1, characterized in that, The step of determining the focal length parameter corresponding to the flame target based on the first detected image includes: Obtain the image size of the first detected image; Based on the pre-trained flame target detection model and the first detection image, the anchor frame size of the target anchor frame corresponding to the flame target is determined; Based on the anchor frame size and the image size of the first detection image, determine the size proportion of the flame target in the first detection image; Based on the size ratio, the focal length parameter corresponding to the flame target is determined.

3. The method according to claim 2, characterized in that, The flame target detection model was trained in the following way: Collect videos corresponding to different scenarios; Images containing flame targets are acquired from each of the videos to obtain a valid dataset; The valid dataset is augmented to obtain an expanded dataset; The initial flame target detection model is iteratively trained based on the expanded dataset until the loss function value of the initial flame target detection model meets the preset training termination condition. The initial flame target detection model at the end of training is then determined as the flame target detection model, wherein the model structure of the initial flame target detection model is the Yolov5x model structure.

4. The method according to any one of claims 1 to 3, characterized in that, Determining the target area where the flame target is located based on the focal length parameter includes: Based on the focal length parameter, a set number of video frames are randomly acquired from the flame target within a set time period to obtain multiple target video frames; For each target video frame, based on the trained flame target detection model, it is detected whether the target video frame is a flame image, wherein the flame image is an image containing a flame target; If the number of detected flame images is greater than a set value, the flame target detection model and the flame images are used to determine the target bounding box corresponding to the flame target in the flame image, and the area corresponding to the target bounding box is determined as the target area where the flame target is located. If the number of detected flame images is not greater than the set value, then images from different regions are acquired until a first detected image with the flame target's location as the image center is acquired again.

5. A fire point positioning device, characterized in that, For implementing the method of any one of claims 1 to 4, comprising: The acquisition module is used to acquire a first detection image with the location of the flame target as the image center, including: Acquire a second detection image containing the flame target; Obtain the center point position of the flame target in the second detected image to obtain the first center point position; Obtain the center point position of the second detected image to obtain the second center point position; Detect whether the positions of the first center point and the second center point are the same; If the position of the first center point is the same as the position of the second center point, then the second detected image is determined as the first detected image; If the position of the first center point is different from the position of the second center point, the first shooting angle corresponding to the second detection image is obtained. Based on the distance information between the position of the first center point and the position of the second center point, the rotation angle of the acquisition device is determined. Based on the first shooting angle and the rotation angle, the second shooting angle corresponding to the first detection image is determined. According to the second shooting angle, the flame target is image acquired to obtain the first detection image. The first processing module is used to determine the focal length parameter corresponding to the flame target based on the first detected image; The second processing module is used to determine the target area where the flame target is located based on the focal length parameter; The positioning module is used to determine the location of a fire point in the target area using a thermal infrared emitter, including: A thermal image of the target area is acquired using a thermal infrared emitter, wherein the deflection angle of the thermal infrared emitter corresponds to different pixels at different locations in the thermal image. Obtain target points in the thermal image that meet set conditions, wherein the set conditions are that the gray value of a pixel is equal to the gray value of the target pixel in the thermal image, and the gray value of each pixel in the set surrounding area corresponding to the target pixel is greater than the set gray value, wherein the target pixel is the pixel corresponding to the largest gray value among all pixels in the thermal image. Multiple laser ranging measurements are performed on the target point to obtain multiple target distances between the target point and the thermal infrared emitter. Based on the respective target distances, the location of the fire point in the target area is determined, including: Determine the average value of each of the target distances, and use the average value as the fire point distance; Obtain the target deflection angle of the thermal infrared emitter corresponding to the target point; Based on the pre-established self-coordinate system, the fire point distance, and the target deflection angle, the first coordinate of the fire point in the self-coordinate system is determined; The first coordinate is converted into a second coordinate in the world coordinate system, and the latitude, longitude and altitude coordinates corresponding to the second coordinate are obtained to obtain the fire point location in the target area; The target deflection angle includes the horizontal plane rotation angle, the side plane rotation angle, and the vertical rotation angle. The origin of the self-coordinate system is the position of the thermal infrared emitter. The first coordinate is determined by the following formula: x h = 0 + l*sinα y *cosα z y h = 0 + l*sin a x *cos a z z h = 0 + l*sin a x *cos a y Where, x h y h and z h The first coordinate represents the coordinates of the first coordinate along the x, y, and z axes of its own coordinate system, l represents the distance to the fire point, and α represents the distance to the fire point. x α represents the angle of rotation in the horizontal plane. y α represents the included angle of rotation of the lateral plane. z Indicates the included angle of vertical rotation.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the steps of the fire point location method as described in any one of claims 1 to 4.

7. An electronic device comprising a memory, a processor, and a program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the fire point location method as described in any one of claims 1 to 4.

Images