A robot-based fire-fighting equipment detection method, medium and robot

Abstract

The application provides a kind of fire-fighting equipment detection method based on robot, S1, obtains fire-fighting cabinet picture;S2, obtains the slope of the horizontal oblique line of fire-fighting cabinet in the picture, and adjusts the negative direction of the slope to move to obtain the second picture;S3, using fire-fighting equipment detection model detects the fire-fighting equipment in the second picture to obtain the first fire-fighting equipment set, wherein the fire-fighting equipment detection model is: the backbone of YOLOv3 is mobilenetv3;S4, whether the first fire-fighting equipment set is short of fire-fighting equipment according to the second fire-fighting equipment set set in advance is judged.The application uses robot to detect fire-fighting equipment, saves manpower.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a robot-based fire-fighting equipment detection method, medium and robot. BACKGROUND

[0002] The placement site of the fire-fighting cabinet should be rainproof and waterproof, and should be more than 1 meter away from the fire source, water source, vibration source and the like, and the setting should be scientific and reasonable, and the distribution should be "key protection, overall consideration", which determines the wide distribution of the fire-fighting cabinet. Secondly, the fire-fighting cabinet is a special cabinet for placing fire-fighting and flood control facilities and equipment, and the main function is to place commonly used fire-fighting equipment such as fire-fighting clothes, fire-fighting helmets, fire-fighting shoes, escape ropes, fire axes, fire extinguishers, fire hoses and gas masks, which determines the diversity of the fire-fighting equipment, and it takes a certain amount of time and labor cost to check whether the fire-fighting equipment is complete. Generally, the inspection of fire-fighting equipment is based on manual inspection, and the corresponding fire-fighting cabinet is found by manual inspection, and the fire-fighting equipment in the fire-fighting cabinet is checked by registration. However, due to the wide distribution of fire-fighting cabinets and the diversity of equipment, manual inspection requires a large amount of time and labor cost. Another way is to use fixed cameras to check: the combination of cameras and manual inspection of fire-fighting equipment. This scheme uses several fixed cameras to shoot fire-fighting cabinets, and then defines fixed area blocks in the shooting area, and checks whether the fire-fighting equipment in the fire-fighting cabinet in the area block is complete by manually watching the images taken at regular intervals. However, the cost of equipment will increase with the increase of cameras; the coverage rate of fire-fighting detection is very limited, and it is difficult to shoot all fire-fighting cabinets; the overall cost and maintenance cost of the system are relatively high.

[0003] The background description provided herein is for the purpose of generally presenting the context of the disclosure. Unless otherwise indicated herein, the subject matter discussed in this section is not prior art to the claims of this application and is not admitted to be prior art by inclusion in this section. SUMMARY

[0004] In view of the above technical problems in the related art, the present application provides a robot-based fire-fighting equipment detection method, which comprises the following steps:

[0005] S1, obtaining a fire-fighting cabinet picture;

[0006] S2, obtaining the slope of the horizontal straight line of the fire-fighting cabinet in the picture, and adjusting the movement of the inspection robot along the negative direction of the slope to obtain a second picture;

[0007] S3, detecting the fire-fighting equipment in the second picture using a fire-fighting equipment detection model to obtain a first fire-fighting equipment set, wherein the fire-fighting equipment detection model is to use mobilenetv3 as the backbone of YOLOv3.

[0008] S4, judging whether the fire-fighting equipment is lacking according to the first fire-fighting equipment set and the second fire-fighting equipment set.

[0009] Specifically, the step 2 specifically comprises:

[0010] S21, acquiring a horizontal straight line in the picture through Hough straight line detection, and acquiring a slope of the horizontal straight line;

[0011] S22, acquiring intersection points outside line segments according to equations of the detected straight lines, and acquiring a plurality of rectangular frames according to compensation points obtained by the intersection points of the line segments and the intersection points of the straight line equations.

[0012] Specifically, the first fire-fighting equipment set is fire-fighting equipment possessed by a fire-fighting cabinet at different positions, or a picture of a fire-fighting cabinet at a specific position is uploaded, and the picture is placed with fire-fighting equipment that should be placed.

[0013] Specifically, the fire-fighting equipment detection model is used to detect fire-fighting equipment in the second picture to acquire the first fire-fighting equipment set in the step S3, specifically, the fire-fighting equipment detection model is used to detect fire-fighting equipment in a picture composed of the rectangular frames to acquire the first fire-fighting equipment set.

[0014] In a second aspect, another embodiment of the present application discloses a fire-fighting equipment detection device based on a robot, which comprises the following units:

[0015] A fire-fighting cabinet picture acquisition unit is configured to acquire a fire-fighting cabinet picture.

[0016] A picture correction unit is configured to acquire a slope of a horizontal straight line of a fire-fighting cabinet in the picture, and adjust movement of a patrol robot in a negative direction of the slope to acquire a second picture.

[0017] A first fire-fighting equipment set acquisition unit is configured to use a fire-fighting equipment detection model to detect fire-fighting equipment in the second picture to acquire a first fire-fighting equipment set, wherein the fire-fighting equipment detection model is that a backbone network backbone of YOLOv3 is mobilenetv3.

[0018] A fire-fighting equipment lacking unit is configured to judge whether the fire-fighting equipment is lacking according to the first fire-fighting equipment set and a second fire-fighting equipment set set in advance.

[0019] Specifically, the picture correction unit further comprises:

[0020] A horizontal straight line slope acquisition unit is configured to acquire a horizontal straight line in the picture through Hough straight line detection, and acquire a slope of the horizontal straight line.

[0021] The rectangular frame acquisition unit is configured to acquire intersection points outside the line segment according to the equation of the detected straight line, and acquire a plurality of rectangular frames according to the intersection points of the line segment and the intersection points of the straight line equation.

[0022] Specifically, the first set of fire-fighting equipment is the fire-fighting equipment possessed by the fire-fighting cabinet at different positions, or the picture of the fire-fighting cabinet at a specific position is uploaded, and the picture is placed with the fire-fighting equipment that should be placed.

[0023] Specifically, the first set of fire-fighting equipment is acquired by detecting the fire-fighting equipment in the second picture using the fire-fighting equipment detection model, specifically, the first set of fire-fighting equipment is acquired by detecting the fire-fighting equipment in the picture composed of the rectangular frame using the fire-fighting equipment detection model.

[0024] In a third aspect, another embodiment of the present application provides a non-volatile memory, which stores instructions, and the instructions are executed by a processor to implement the above-mentioned fire-fighting equipment detection method based on a robot.

[0025] In a fourth aspect, another embodiment of the present application provides a robot, which comprises a processing module, a chassis, a storage module, and a camera, and the storage module stores instructions, and the instructions are executed to implement the above-mentioned fire-fighting equipment detection method based on a robot.

[0026] The present application uses a robot to detect and analyze all the planned fire-fighting cabinets, which not only reduces the cost of the camera, but also greatly improves the coverage of the detection, and provides greater support for commercialization. The input image is preprocessed using the Hough line detection principle, which can not only guide the path of the inspection robot and the angle of the camera, but also shoot high-quality images, and obtain intersection points according to the straight line equation, thereby obtaining a plurality of rectangular frames, and improving the detection accuracy. The collected images are automatically analyzed using a deep learning scheme, and the training data set is masked to a certain extent to improve the generalization ability of the model. Secondly, the model is improved, and a lightweight network is used, which has good robustness and high precision. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0028] Figure 1 is a schematic diagram of the fire-fighting equipment detection system based on a robot provided by the embodiments of the present application.

[0029] Figure 2 is a robot-based fire-fighting equipment detection method provided by an embodiment of the present application;

[0030] Figure 3 is a photographed front view of a fire-fighting cabinet provided by an embodiment of the present application;

[0031] Figure 4 is a photographed fire-fighting cabinet provided by an embodiment of the present application;

[0032] Figure 5 is a corrected fire-fighting cabinet provided by an embodiment of the present application;

[0033] Figure 6 is a robot-based fire-fighting equipment detection device provided by an embodiment of the present application;

[0034] Figure 7 is a robot-based fire-fighting equipment detection device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0036] Embodiment One

[0037] With reference to Figure 2 , the present embodiment discloses a robot-based fire-fighting equipment detection method, which comprises the following steps:

[0038] S1, obtaining a fire-fighting cabinet picture;

[0039] With specific reference to Figure 1 , Figure 1 is a schematic diagram of a robot-based fire-fighting equipment detection system of the present embodiment, in which the camera on the robot is always directed towards the fire-fighting cabinet 1; the large dot represents the position of the robot; and the arrow straight line represents the inspection route of the robot. Since the inspection route is fixed, and the camera shooting angle is also directly towards the fire-fighting cabinet, the images photographed are basically front pictures of the fire-fighting cabinet, as shown in Figure 3 .

[0040] Specifically, the robot of the present embodiment has a camera, which is placed on the head of the robot, and continuously patrols the area to capture images to obtain the fire-fighting cabinet picture. The capturing method can be a continuous capturing method to form a video stream, or a fixed-point capturing method.

[0041] S2, obtaining the slope of the horizontal straight line of the fire-fighting cabinet in the picture, and adjusting the patrol robot to move in the negative direction of the slope to obtain a second picture;

[0042] Specifically, the robot patrol can customize a corresponding patrol route according to the layout of the area, but when grabbing the fire-fighting cabinet, it cannot always keep the camera frontally shooting the fire-fighting cabinet. Referring to Figure 4 , Figure 4 is a picture of a fire-fighting cabinet grabbed, from Figure 4 it can be seen that the grabbed fire-fighting cabinet picture is not a frontal picture, and the horizontal straight line of the fire-fighting cabinet has a slope.

[0043] This embodiment uses the Hough straight line detection principle to detect the straight line in the picture, analyzes the equation of the detected straight line, guides the motion trajectory and shooting angle of the patrol robot, and thus obtains a high-quality image.

[0044] S21, obtaining the horizontal straight line in the picture by Hough straight line detection, and obtaining the slope of the horizontal straight line;

[0045] Specifically, Hough straight line detection belongs to a common straight line detection algorithm in the art, which can output the angle of the straight line.

[0046] Referring to Figure 4 , this embodiment uses Hough straight line detection to obtain the straight line in the picture and obtain the slope of the horizontal straight line.

[0047] Then the patrol robot is adjusted to move in the negative direction of the slope to obtain a picture of the fire-fighting cabinet until an angle is obtained, which is a picture directly facing the fire-fighting cabinet, which improves the accuracy of fire-fighting cabinet and fire-fighting equipment detection.

[0048] S22, obtaining intersection points outside the line segment according to the equation of the detected straight line, and obtaining a plurality of rectangular frames according to the intersection points of the line segment and the intersection points of the straight line equation;

[0049] After recognizing the approximate position of the front of the fire-fighting cabinet, the image of the fire-fighting cabinet is detected according to the Hough straight line detection principle, and the intersection points outside the line segment are obtained according to the equation of the straight line, as shown in Figure 5 the line is the line segment detected by the Hough straight line detection principle, and the point is the compensation point based on the equation of the straight line, i.e. the intersection point outside the line segment. According to the compensation point obtained by the intersection points of the line segment and the intersection points of the straight line equation, a plurality of rectangular frames are obtained after analysis, and are sent into the fire-fighting equipment detection model. The advantage of this is that the frame required by this embodiment can be positioned more accurately, and at the same time the fire-fighting cabinet occupies the entire picture as much as possible, improving the recognition accuracy of the fire-fighting equipment.

[0050] S3, detecting fire-fighting equipment in the second picture using a fire-fighting equipment detection model to obtain a first fire-fighting equipment set, wherein the fire-fighting equipment detection model is to use a backbone network of YOLOv3 as mobilenetv3;

[0051] The embodiment uses fire-fighting cabinet data collected by an inspection robot and fire-fighting equipment data artificially shot to generate a data set, and performs rectangular frame labeling on the fire-fighting equipment on the basis of the data set. Five data are generated for each equipment, which are respectively a category and a position, that is, label represents the category, (x1, y1) is the upper left corner of the equipment frame, and (x2, y2) is the lower right corner of the equipment frame. Since the fire-fighting equipment is stored in a glass cover, and the text on the glass is different for different fire-fighting cabinets, the artificially shot fire-fighting equipment is masked to a certain extent, and a certain Gaussian noise is added. This has the advantage of improving the generalization ability of the recognition model.

[0052] After the image is obtained, the embodiment uses a deep learning target detector YOLOv3 to perform training. In order to improve the inference speed, the backbone of YOLOv3 is modified to mobilenetv3, so that real-time effect can be achieved on an edge platform. The training data of the YOLOv3 model is collected and labeled by the embodiment. In order to improve the universality and ability of the model, the focus structure in YOLOv3 is removed, and the picture input for training and inference of the model is changed to the resolution ratio of a video to become 16:9 for multiscale training. Since the fire-fighting cabinet and the fire-fighting equipment have certain similarities in structure and color channels, in the features extracted by mobilinet V3, the feature maps are first fused by a channel and spatial attention mechanism, and then transmitted to a YOLOv3 recognition module for recognition.

[0053] S4, judging whether fire-fighting equipment is missing according to the first fire-fighting equipment set and a second fire-fighting equipment set set in advance;

[0054] Different fire-fighting cabinets store different fire-fighting equipment. In order to meet different scenes, the embodiment needs to pre-record the equipment categories of the fire-fighting cabinets.

[0055] The embodiment can pre-record the fire-fighting equipment possessed by the fire-fighting cabinets at different positions by manual input, or upload pictures of the fire-fighting cabinets at specific positions. The pictures require that the fire-fighting equipment that should be placed is placed.

[0056] The path of the inspection robot is planned, and the fire-fighting cabinet in a specific position can be easily obtained. Therefore, the embodiment can upload a specific fire-fighting cabinet picture. After the system loads the picture, the robot will capture the fire-fighting cabinet under the specific path during the inspection process. After identifying and obtaining the fire-fighting equipment, the system will judge whether the equipment is missing. If the equipment is missing, the inspection robot will issue an alarm.

[0057] Specifically, the robot can send the corresponding fire-fighting cabinet with missing equipment to the server. The staff of the server can handle the alarm information, such as notifying personnel to supplement the missing fire-fighting equipment.

[0058] The embodiment uses the robot inspection method to detect and analyze all planned fire-fighting cabinets. This not only reduces the cost of the camera, but also greatly improves the coverage of the detection, providing greater support for commercialization. Using the Hough line detection principle for image preprocessing can guide the path and shooting angle of the inspection robot, capture high-quality images, and obtain intersection points from the straight line equation to obtain several rectangular frames, thereby improving the detection accuracy. The deep learning scheme is used to automatically analyze the collected images. The training data set is masked to a certain extent to improve the generalization ability of the model. Secondly, the model is improved using a lightweight network with good robustness and high precision.

[0059] Embodiment Two

[0060] Reference Figure 6 The embodiment discloses a robot-based fire-fighting equipment detection device, which comprises the following units:

[0061] A fire-fighting cabinet picture acquisition unit is configured to acquire fire-fighting cabinet pictures.

[0062] For details, please refer to Figure 1 , Figure 1 is a schematic diagram of a robot-based fire-fighting equipment detection system of the embodiment. The camera on the robot is directed towards the fire-fighting cabinet 1. The large dots represent the positions of the robot, and the arrow lines represent the inspection routes of the robot. Since the inspection route is fixed, and the camera shooting angle is also directly opposite the fire-fighting cabinet, the images captured are basically front pictures of the fire-fighting cabinet, as shown in Figure 3 .

[0063] Specifically, the robot of the embodiment has a camera, which is placed on the head of the robot. The camera continuously captures images by constantly patrolling the area to obtain the fire-fighting cabinet pictures. The capturing method can be continuous capturing to form a video stream, or it can be a fixed-point capturing method.

[0064] a picture correction unit configured to acquire a slope of a horizontal straight line of the fire-fighting cabinet in the picture, and adjust the patrol robot to move in a negative direction of the slope to acquire a second picture;

[0065] Specifically, the robot patrol can customize a corresponding patrol route according to the layout of the area, but when grabbing the fire-fighting cabinet, it cannot always keep the camera frontally shooting the fire-fighting cabinet. Figure 4 , Figure 4 is a picture of a fire-fighting cabinet grabbed by the robot patrol. Figure 4 It can be seen that the grabbed picture of the fire-fighting cabinet is not a frontal picture, and the horizontal straight line of the fire-fighting cabinet has a slope.

[0066] This embodiment uses the Hough straight line detection principle to detect the straight line in the picture, analyzes the equation of the detected straight line, and guides the motion trajectory and shooting angle of the patrol robot, so as to obtain a high-quality image.

[0067] a horizontal straight line slope acquisition unit configured to acquire a horizontal straight line in the picture through the Hough straight line detection, and acquire a slope of the horizontal straight line;

[0068] Specifically, the Hough straight line detection belongs to a common straight line detection algorithm in the art, which can output the angle of the straight line.

[0069] Referring to Figure 4 , this embodiment uses the Hough straight line detection to acquire the straight line in the picture and acquire the slope of the horizontal straight line.

[0070] Then the patrol robot is adjusted to move in a negative direction of the slope to acquire a picture of the fire-fighting cabinet until an angle of a picture is directly opposite to the fire-fighting cabinet, which improves the accuracy of fire-fighting cabinet and fire-fighting equipment detection.

[0071] a rectangular frame acquisition unit configured to acquire intersection points outside the line segment according to the equation of the detected straight line, acquire compensation points according to the intersection points of the line segment and the intersection points of the straight line equation, and acquire a plurality of rectangular frames;

[0072] After the approximate position of the front of the fire-fighting cabinet is recognized, the image of the fire-fighting cabinet is detected according to the Hough straight line detection principle, and the intersection points outside the line segment are obtained according to the equation of the straight line, as shown in Figure 5 The line is a line segment detected by the Hough straight line detection principle, and the point is a compensation point based on the equation of the straight line, i.e., an intersection point outside the line segment. Compensation points are acquired according to the intersection points of the line segment and the intersection points of the straight line equation, and a plurality of rectangular frames are obtained after analysis and sent to the fire-fighting equipment detection model. The advantage of this is that the frame required by this embodiment can be positioned more accurately, and the fire-fighting cabinet can occupy the entire picture as much as possible, improving the recognition accuracy of the fire-fighting equipment.

[0073] The first fire-fighting equipment set acquisition unit is configured to acquire a first fire-fighting equipment set by detecting fire-fighting equipment in the second picture using a fire-fighting equipment detection model, wherein the fire-fighting equipment detection model is a model in which a backbone of YOLOv3 is replaced by mobilenetv3.

[0074] The embodiment uses fire-fighting cabinet data collected by an inspection robot and fire-fighting equipment data artificially shot to generate a data set, and performs rectangular frame labeling on the fire-fighting equipment on the basis of the data set. Five data are generated for each equipment, which are respectively a category and a position, that is, label represents the category, (x1, y1) is the upper left corner of the equipment frame, and (x2, y2) is the lower right corner of the equipment frame. Since the fire-fighting equipment is stored in a glass cover, and the text on the glass is different for different fire-fighting cabinets, the artificially shot fire-fighting equipment is masked to a certain extent, and a certain Gaussian noise is added. This has the advantage of improving the generalization ability of the recognition model.

[0075] After the image is obtained, the embodiment uses a deep learning target detector YOLOv3 to perform training. In order to improve the inference speed, the backbone of YOLOv3 is modified to mobilenetv3, so that real-time effect can be achieved on an edge platform. The training data of the YOLOv3 model is collected and labeled by the embodiment. In order to improve the universality and ability of the model, the focus structure in YOLOv3 is removed, and the picture input for training and inference of the model is changed to a multiscale training with a resolution ratio of 16:9. Since the fire-fighting cabinet and the fire-fighting equipment have certain similarities in structure and color channels, in the features extracted by mobilinet V3, the channel and spatial attention mechanism are first fused, and then the features are transmitted to the YOLOv3 recognition module for recognition.

[0076] The fire-fighting equipment deficiency unit is configured to determine whether fire-fighting equipment is deficient according to the first fire-fighting equipment set and a second fire-fighting equipment set set in advance.

[0077] Different fire-fighting cabinets store different fire-fighting equipment. In order to meet different scenes, the embodiment needs to pre-record the equipment categories of the fire-fighting cabinets.

[0078] The embodiment can pre-record the fire-fighting equipment possessed by the fire-fighting cabinets at different positions by manual input, or upload pictures of the fire-fighting cabinets at specific positions. The pictures require that the fire-fighting equipment that should be placed is placed.

[0079] Since the path of the inspection robot is planned, it is easy to obtain the fire cabinet at a specific position, so this embodiment can upload a specific fire cabinet picture, and after the system loads the picture, the robot will capture the fire cabinet under the specific path in the inspection process. After identifying and obtaining the fire equipment, it is judged with the system loaded category. If it is lacking, the inspection robot will issue an alarm.

[0080] This embodiment uses the robot inspection method to detect and analyze all planned fire cabinets, which not only reduces the cost of the camera, but also greatly improves the coverage of the detection, providing greater support for commercialization. Using the Hough line detection principle for pre-processing of the input image, not only can guide the path of the inspection robot and the shooting angle, shoot high-quality images, but also obtain the intersection according to the straight line equation, thereby obtaining several rectangular frames, and improve the detection accuracy. A deep learning scheme is used to automatically analyze the collected images, and a certain degree of mask is performed on the training data set to improve the generalization ability of the model. Secondly, the model is improved, and a lightweight network is used, which has good robustness and high precision.

[0081] Embodiment three

[0082] The embodiment discloses a robot, which comprises a processing module, a chassis, a storage module and a camera. The storage module stores instructions which, when executed, implement the robot-based fire equipment detection method of embodiment one.

[0083] Specifically, the robot of the embodiment further comprises the robot-based fire equipment detection device of embodiment two.

[0084] Embodiment four

[0085] Reference Figure 7 , Figure 7 is a structural schematic diagram of a robot-based fire equipment detection device of the embodiment. The robot-based fire equipment detection device 20 of the embodiment comprises a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. The processor 21 executes the computer program to implement the steps in the above method embodiments. Alternatively, the processor 21 executes the computer program to implement the functions of each module / unit in the above device embodiments.

[0086] For example, the computer program can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the robot-based fire-fighting equipment detection device 20. For example, the computer program can be divided into the modules in Embodiment Two, and the specific functions of the modules can refer to the working process of the device described in the above embodiments, which will not be described here.

[0087] The robot-based fire-fighting equipment detection device 20 can include, but is not limited to, the processor 21 and the memory 22. Those skilled in the art can understand that the schematic diagram is only an example of the robot-based fire-fighting equipment detection device 20, and does not constitute a limitation on the robot-based fire-fighting equipment detection device 20, which can include more or fewer components than the diagram, or combine certain components, or different components, for example, the robot-based fire-fighting equipment detection device 20 can also include an input / output device, a network access device, a bus, etc.

[0088] The processor 21 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor 21 is the control center of the robot-based fire-fighting equipment detection device 20, which connects all parts of the robot-based fire-fighting equipment detection device 20 through various interfaces and lines.

[0089] The memory 22 can be used to store the computer programs and / or modules, and the processor 21 realizes various functions of the robot-based fire-fighting equipment detection device 20 by running or executing the computer programs and / or modules stored in the memory 22, and calling the data stored in the memory 22. The memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0090] When the modules / units of the robot-based fire-fighting equipment detection device 20 are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor 21, the steps of the above-mentioned various method embodiments can be realized. The computer program includes computer program code, which can be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the computer-readable medium can include or exclude contents according to the requirements of legislation and patent practice in a jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0091] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate units can or can not be physically separate, and the units shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. In addition, the connection relationship between the modules in the apparatus embodiment provided by the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0092] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A robot-based fire-fighting equipment detection method, comprising the following steps: S1.Obtaining a fire cabinet picture by a camera on a robot head in a manner of continuous snapshot forming a video stream or fixed-point snapshot; S2, obtaining the slope of the horizontal oblique straight line of the fire-fighting cabinet in the picture, and adjusting the patrol robot to move in a direction to reduce the inclination of the fire-fighting cabinet picture to obtain a second picture, the second picture being a front image of the fire-fighting cabinet; specifically comprising: Obtaining a horizontal oblique straight line in the picture by Hough line detection and obtaining a slope of the horizontal oblique straight line; Detecting the second picture according to the Hough line detection to obtain a straight line equation, obtaining intersection points outside line segments according to the detected straight line equation, obtaining compensation points according to the intersection points of the line segments and the intersection points outside the line segments, and then obtaining a plurality of rectangular frames; S3.Detecting fire-fighting equipment in the second picture by using a fire-fighting equipment detection model to obtain a first fire-fighting equipment set, specifically, detecting fire-fighting equipment in a picture composed of the rectangular frames by using the fire-fighting equipment detection model to obtain the first fire-fighting equipment set, wherein the fire-fighting equipment detection model is: taking a backbone of YOLOv3 as mobilenetv3, adjusting a resolution ratio of an input picture of model training and reasoning to a 16:9 ratio for multiscale training, and in features extracted by mobilenetv3, first fusing channel and spatial attention mechanisms of the extracted feature maps, and then transmitting the feature maps to a YOLOv3 recognition module for recognition; S4.Determining whether fire-fighting equipment is missing according to the first fire-fighting equipment set and a second fire-fighting equipment set set in advance, wherein the second fire-fighting equipment set is fire-fighting equipment information possessed by a fire cabinet at different positions recorded in advance, or a fire cabinet picture of a specific position and placed with fire-fighting equipment that should be placed, wherein the picture is placed with fire-fighting equipment that should be placed. 2.A robot-based fire-fighting equipment detection device, comprising the following units: A fire cabinet picture obtaining unit, configured to obtain a fire cabinet picture by a camera on a robot head in a manner of continuous snapshot forming a video stream or fixed-point snapshot; The picture correction unit is configured to acquire a slope of a horizontal oblique line of the fire-fighting cabinet in the picture, and adjust the movement direction of the inspection robot to reduce the inclination of the picture of the fire-fighting cabinet to obtain a second picture, which is a front image of the fire-fighting cabinet. A horizontal straight line slope obtaining unit, configured to obtain a horizontal oblique straight line in the picture by Hough line detection and obtain a slope of the horizontal oblique straight line; A rectangular frame obtaining unit, configured to detect the second picture according to the Hough line detection to obtain a straight line equation, obtain intersection points outside line segments according to the detected straight line equation, obtain compensation points according to the intersection points of the line segments and the intersection points outside the line segments, and obtain a plurality of rectangular frames; The first fire-fighting equipment set acquisition unit is configured to acquire a first fire-fighting equipment set by detecting fire-fighting equipment in the second picture using a fire-fighting equipment detection model, specifically, acquire a first fire-fighting equipment set by detecting fire-fighting equipment in a picture formed by the rectangular frame using a fire-fighting equipment detection model, wherein the fire-fighting equipment detection model is as follows: the backbone of YOLOv3 is mobilenetv3, the resolution ratio of an input picture of model training and inference is adjusted to a multiscale training of a proportion of 16:9, and in the features extracted by mobilenetv3, the extracted feature maps are first fused by a channel and spatial attention mechanism, and then transmitted to a YOLOv3 recognition module for recognition; The fire-fighting equipment deficiency unit is configured to determine whether fire-fighting equipment is deficient according to the first fire-fighting equipment set and a second fire-fighting equipment set set in advance, wherein the second fire-fighting equipment set is fire-fighting equipment information possessed by fire-fighting cabinets at different positions recorded in advance, or a fire-fighting cabinet picture uploaded for a specific position and placed with fire-fighting equipment that should be placed, wherein the picture is placed with fire-fighting equipment that should be placed.

3. A non-volatile memory, wherein the memory stores instructions, and the instructions are executed by a processor to implement the robot-based fire-fighting equipment detection method according to claim 1.

4. A robot comprising: A processing module, a chassis, a storage module, and a camera, wherein the storage module stores instructions, and the instructions are executed to implement the robot-based fire-fighting equipment detection method according to claim 1.

Images