Image recognition method for fire fighting robot

Abstract

The embodiment of the invention provides an image recognition method for a fire-fighting robot, and belongs to the technical field of fire-fighting robot image recognition. The image recognition method comprises the following steps: acquiring a historical image of the fire-fighting robot; acquiring a real-time image of the current fire-fighting robot; and according to the real-time image and the historical image, identifying the fire-fighting state of the equipment in the current transformer substation. According to the image recognition method for the fire-fighting robot, the historical image of the fire-fighting robot is obtained firstly, then the current real-time image of the fire-fighting robot is collected, and the real-time image is compared with the historical image to judge whether a fire exists in the current real-time image or not; and thus, the accuracy and timeliness of monitoring the fire behavior of the transformer substation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fire-fighting robot image recognition, and in particular to an image recognition method for a fire-fighting robot. Background Art

[0002] With the popularization of intelligent robots, fire-fighting robots are gradually being used in substations for fire inspections of substations.

[0003] At present, firefighting robots need to identify the status of the substation site during inspections, generally using smoke sensors or background monitoring. However, this method cannot capture distant fires in time, resulting in the fire being unable to be dealt with in a timely manner, affecting the stability and safety of substation operation.

[0004] The inventors of the present application discovered during the process of implementing the present invention that the above-mentioned solutions in the prior art have the defect of poor timeliness in fire identification. Summary of the invention

[0005] An object of an embodiment of the present invention is to provide an image recognition method for a fire extinguisher robot, which has a function of good recognition timeliness.

[0006] In order to achieve the above object, an embodiment of the present invention provides an image recognition method for a fire fighting robot, comprising:

[0007] Obtain historical images of firefighting robots;

[0008] Collect real-time images of the current firefighting robot;

[0009] The fire protection status of the equipment in the current substation is identified according to the real-time image and the historical image.

[0010] Optionally, obtaining the historical image of the firefighting robot includes:

[0011] Driving the fire-fighting robot to move along a fire-fighting trajectory;

[0012] Obtaining a video captured by a camera of the fire-fighting robot during movement of the fire-fighting track;

[0013] The video is split according to preset time intervals to obtain multiple groups of historical images of equipment in the substation.

[0014] Optionally, acquiring historical images of the firefighting robot further includes:

[0015] Preset multiple groups of device scenes;

[0016] Allocating the plurality of groups of historical images equally to the plurality of groups of device scenes;

[0017] Each group of the device scenarios is numbered and named.

[0018] Optionally, identifying the fire protection status of equipment in the current substation according to the real-time image and the historical image includes:

[0019] Obtain the device scene where the real-time image is located and the corresponding label and scene name;

[0020] Retrieving a plurality of historical images in the device scene corresponding to the real-time image;

[0021] Comparing the real-time image with the plurality of historical images in terms of pixels to obtain similarities between the real-time image and the plurality of historical images;

[0022] The fire protection status of the equipment in the current substation is determined according to the similarity.

[0023] Optionally, obtaining the device scene in which the real-time image is located and the corresponding label and scene name includes:

[0024] Obtain the current position of the fire-fighting robot;

[0025] Obtaining a number according to the current position of the fire-fighting robot;

[0026] The equipment scene in which the fire-fighting robot is currently located is obtained according to the label.

[0027] Optionally, performing pixel comparison between the real-time image and the plurality of historical images to obtain similarities between the real-time image and the plurality of historical images includes:

[0028] Acquire a feature vector of the real-time image;

[0029] Acquire feature vectors of a plurality of the historical images in the device scene corresponding to the real-time image;

[0030] The similarity between the real-time image and the plurality of historical images is calculated according to formula (1).

[0031]

[0032] Among them, SIM is the similarity between the real-time image and the historical image, A is the feature vector of the real-time image, B i is the feature vector of the i-th historical image, where i is an integer number.

[0033] Optionally, determining the fire protection status of equipment in the current substation according to the similarity includes:

[0034] Determining whether the similarity is greater than or equal to a similarity threshold;

[0035] In the case where it is determined that the similarity is greater than or equal to the similarity threshold, determining that the device in the device scene is normal;

[0036] When it is determined that the similarity is less than the similarity threshold, it is determined that the equipment in the equipment scene has a fire hazard.

[0037] Optionally, determining the fire protection status of equipment in the current substation according to the similarity includes:

[0038] Determining whether the similarity is greater than or equal to a first similarity threshold;

[0039] In the case where it is determined that the similarity is greater than or equal to the first similarity threshold, determining that the device in the current device scene is normal;

[0040] In the case where it is determined that the similarity is less than the first similarity, determining whether the similarity is greater than or equal to a second similarity;

[0041] In the case where it is determined that the similarity is greater than or equal to a second similarity threshold, determining that the equipment in the current equipment scene has a fire hazard;

[0042] When it is determined that the similarity is less than the second similarity threshold, it is determined that a fire accident occurs in the equipment in the current equipment scene, and an audible and visual alarm is issued.

[0043] Through the above technical scheme, the image recognition method for the fire-fighting robot provided by the present invention first obtains the historical image of the fire-fighting robot, then collects the current real-time image of the fire-fighting robot, and compares the real-time image with the historical image to determine whether there is a fire in the current real-time image, thereby improving the accuracy and timeliness of monitoring the substation fire. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0045] Figure 1 is a flow chart of an image recognition method for a fire fighting robot according to one embodiment of the present invention;

[0046] Figure 2 is a flow chart of classifying equipment scenes in an image recognition method for a fire fighting robot according to an embodiment of the present invention;

[0047] Figure 3 is a flow chart of determining a firefighting state in an image recognition method for a fire-fighting robot according to an embodiment of the present invention;

[0048] Figure 4 is a flow chart of obtaining similarity in an image recognition method for a fire fighting robot according to an embodiment of the present invention;

[0049] Figure 5 is a flow chart of similarity judgment in an image recognition method for a fire fighting robot according to an embodiment of the present invention;

[0050] Figure 6 The present invention is a flowchart of similarity judgment in an image recognition method for a fire-fighting robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0052] Figure 1 FIG. 1 is a flow chart of an image recognition method for a fire fighting robot according to an embodiment of the present invention. Figure 1 In the method, the image recognition method may include:

[0053] In step S10, historical images of the firefighting robot are obtained.

[0054] In step S11, a real-time image of the current firefighting robot is collected.

[0055] In step S12, the fire protection status of the equipment in the current substation is identified based on the real-time image and the historical image.

[0056] In step S10 to step S12, the historical images of the fire-fighting robot in the inspection track are first obtained, and the current real-time image of the fire-fighting robot is collected in real time. The current real-time image is compared with the historical image to determine whether there is smoke, fire obstruction, etc. in the current real-time image, so as to be able to judge whether there is a fire in the real-time image, thereby improving the accuracy and timeliness of monitoring the substation fire.

[0057] When inspecting, traditional firefighting robots need to identify the status of the substation site, generally using smoke sensors or background monitoring. However, this method cannot capture fires in the distance in time, resulting in the inability to deal with fires in time, affecting the stability and safety of substation operation. In this embodiment of the present invention, by comparing real-time images with historical images, it is possible to effectively determine whether there is a fire in the current real-time image, thereby improving the accuracy and timeliness of monitoring the substation fire.

[0058] In this embodiment of the present invention, the acquisition of historical images of the fire-fighting robot can be achieved by driving the fire-fighting robot to move along the fire-fighting track and capturing the video during the movement. The specific acquisition steps can be as follows: Figure 2 Specifically, Figure 2 In the method for image recognition, the image recognition method may further include:

[0059] In step S20, the firefighting robot is driven to move along the firefighting trajectory.

[0060] In step S21, the video captured by the camera of the fire-fighting robot during its movement on the fire-fighting track is obtained.

[0061] In step S22, the video is split according to preset time intervals to obtain multiple groups of historical images of the equipment in the substation.

[0062] In step S23, multiple groups of device scenarios are preset.

[0063] In step S24, multiple groups of historical images are equally distributed to multiple groups of device scenes.

[0064] In step S25, each group of device scenes is numbered and named.

[0065] In this embodiment of the present invention, after obtaining the historical image and the real-time image of the substation equipment, it is necessary to compare the two to obtain the fire protection status of the current equipment in the substation. The specific comparison step can be as follows: Figure 3 Specifically, Figure 3 In the method for image recognition, the image recognition method may further include:

[0066] In step S30, the device scene where the real-time image is located and the corresponding label and scene name are obtained.

[0067] In step S31, a plurality of historical images in the device scene corresponding to the real-time image are retrieved.

[0068] In step S32, the real-time image is compared with the plurality of historical images in terms of pixels to obtain the similarity between the real-time image and the plurality of historical images.

[0069] In step S33, the fire protection status of the equipment in the current substation is determined according to the similarity.

[0070] In step S30 to step S33, the scene of the real-time image is first determined, and then multiple historical images of the same scene are retrieved, and the real-time image is pixel-wise compared with the multiple historical images. The similarity between the real-time image and each historical image is obtained respectively, and then the fire protection status of the equipment in the current substation is determined.

[0071] In this embodiment of the present invention, the scene corresponding to the real-time image can be determined based on the current position of the fire-fighting robot. The specific determination steps may include: first obtaining the current position of the fire-fighting robot, then obtaining a label based on the current position of the fire-fighting robot, and finally obtaining the equipment scene where the current fire-fighting robot is located based on the label.

[0072] In this embodiment of the present invention, when calculating the similarity between the real-time image and each historical image, it is necessary to first obtain the feature vector of the image. The specific steps can be as follows: Figure 4 Specifically, Figure 4 In the method for image recognition, the image recognition method may further include:

[0073] In step S40, a feature vector of the real-time image is obtained.

[0074] In step S41, feature vectors of multiple historical images in the device scene corresponding to the real-time image are obtained.

[0075] In step S42, the similarity between the real-time image and the multiple historical images is calculated according to formula (1):

[0076]

[0077] Among them, SIM is the similarity between the real-time image and the historical image, A is the feature vector of the real-time image, and B i is the feature vector of the i-th historical image, where i is an integer number.

[0078] In this embodiment of the present invention, in order to determine the fire protection status of the equipment in the current substation, it is also necessary to judge the similarity between the real-time image and each historical image. The specific judgment step can be as follows: Figure 5 Specifically, Figure 5 In the method for image recognition, the image recognition method may further include:

[0079] In step S50, it is determined whether the similarity is greater than or equal to a similarity threshold.

[0080] In step S51, when it is determined that the similarity is greater than or equal to the similarity threshold, it is determined that the device in the device scene is normal.

[0081] In step S52, when it is determined that the similarity is less than the similarity threshold, it is determined that the equipment in the equipment scene has a fire hazard.

[0082] In step S50 to step S52, each similarity is compared with a similarity threshold. If the similarity is greater than the similarity threshold, the device is determined to be normal. Otherwise, there is a safety hazard, and a warning is issued and emergency measures are taken.

[0083] In this embodiment of the present invention, in order to determine the fire protection status of the equipment in the current substation, it is also necessary to judge the similarity between the real-time image and each historical image. The specific judgment step can be as follows: Figure 6 Specifically, Figure 6 In the method for image recognition, the image recognition method may further include:

[0084] In step S60, it is determined whether the similarity is greater than or equal to a first similarity threshold.

[0085] In step S61, when it is determined that the similarity is greater than or equal to the first similarity threshold, it is determined that the device in the current device scene is normal.

[0086] In step S62, when it is determined that the similarity is less than the first similarity, it is determined whether the similarity is greater than or equal to the second similarity.

[0087] In step S63, when it is determined that the similarity is greater than or equal to the second similarity threshold, it is determined that the equipment in the current equipment scene has a fire hazard.

[0088] In step S64, when it is determined that the similarity is less than the second similarity threshold, it is determined that a fire accident has occurred in the equipment in the current equipment scene, and an audible and visual alarm is issued.

[0089] In step S60 to step S64, the similarity threshold is divided into two levels. When the similarity is greater than the first similarity threshold, the device is judged to be normal. When the similarity is between the first similarity threshold and the second similarity threshold, it is judged that there may be a fire hazard and an alarm is issued. If the similarity is less than the second similarity threshold, it is directly judged that a fire accident has occurred, a warning is issued, and emergency measures are taken.

[0090] Through the above technical scheme, the image recognition method for the fire-fighting robot provided by the present invention first obtains the historical image of the fire-fighting robot, then collects the current real-time image of the fire-fighting robot, and compares the real-time image with the historical image to determine whether there is a fire in the current real-time image, thereby improving the accuracy and timeliness of monitoring the substation fire.

[0091] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0092] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0093] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0094] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0095] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0096] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0097] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0098] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0099] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An image recognition method for a fire fighting robot, characterized in that: include: Obtain historical images of firefighting robots; Collect real-time images of the current firefighting robot; The fire protection status of the equipment in the current substation is identified according to the real-time image and the historical image.

2. The image recognition method according to claim 1, characterized in that: Obtaining historical images of the firefighting robot includes: Driving the fire-fighting robot to move along a fire-fighting trajectory; Obtaining a video captured by a camera of the fire-fighting robot during movement of the fire-fighting track; The video is split according to preset time intervals to obtain multiple groups of historical images of equipment in the substation.

3. The image recognition method according to claim 2, characterized in that: Acquisition of historical images of firefighting robots also includes: Preset multiple groups of device scenes; Allocating the plurality of groups of historical images equally to the plurality of groups of device scenes; Each group of the device scenarios is numbered and named.

4. The image recognition method according to claim 3, characterized in that: Identifying the fire protection status of the equipment in the current substation according to the real-time image and the historical image includes: Obtain the device scene where the real-time image is located and the corresponding label and scene name; Retrieving a plurality of historical images in the device scene corresponding to the real-time image; Comparing the real-time image with the plurality of historical images in terms of pixels to obtain similarities between the real-time image and the plurality of historical images; The fire protection status of the equipment in the current substation is determined according to the similarity.

5. The image recognition method according to claim 4, characterized in that: The device scene where the real-time image is located and the corresponding label and scene name include: Obtain the current position of the fire-fighting robot; Obtaining a number according to the current position of the fire-fighting robot; The equipment scene in which the fire-fighting robot is currently located is obtained according to the label.

6. The image recognition method according to claim 5, characterized in that: Comparing the real-time image with the plurality of historical images by pixel comparison to obtain the similarity between the real-time image and the plurality of historical images includes: Acquire a feature vector of the real-time image; Acquire feature vectors of a plurality of the historical images in the device scene corresponding to the real-time image; The similarity between the real-time image and the plurality of historical images is calculated according to formula (1). Among them, SIM is the similarity between the real-time image and the historical image, A is the feature vector of the real-time image, B i is the feature vector of the i-th historical image, where i is an integer number.

7. The image recognition method according to claim 6, characterized in that: Determining the fire protection status of the equipment in the current substation according to the similarity includes: Determining whether the similarity is greater than or equal to a similarity threshold; In the case where it is determined that the similarity is greater than or equal to the similarity threshold, determining that the device in the device scene is normal; When it is determined that the similarity is less than the similarity threshold, it is determined that the equipment in the equipment scene has a fire hazard.

8. The image recognition method according to claim 6, characterized in that: Determining the fire protection status of the equipment in the current substation according to the similarity includes: Determining whether the similarity is greater than or equal to a first similarity threshold; In the case where it is determined that the similarity is greater than or equal to the first similarity threshold, determining that the device in the current device scene is normal; In the case where it is determined that the similarity is less than the first similarity, determining whether the similarity is greater than or equal to a second similarity; When it is determined that the similarity is greater than or equal to a second similarity threshold, determining that the equipment in the current equipment scene has a fire hazard; When it is determined that the similarity is less than the second similarity threshold, it is determined that a fire accident occurs in the equipment in the current equipment scene, and an audible and visual alarm is issued.

Images