Fire pump safety identification system and method

Abstract

This invention relates to information processing technology. The purpose of this invention is to solve the problems of current fire pump safety assessment methods, which rely on manual observation leading to misjudgments, or the low sensitivity, susceptibility to environmental interference, and maintenance difficulties associated with physical contact and sensor technologies. This invention provides a fire pump safety identification system and method, the technical solution of which can be summarized as follows: an image acquisition unit is connected to a processing unit; the image acquisition unit collects video information from the inspection process and sends the video information to the processing unit in real time; the processing unit predicts whether the button status of the inspected fire pump is normal based on the video information and a start / stop operation recognition model, and predicts whether the dial of the inspected fire pump is normal based on the video information, a dial detection model, and a pointer segmentation model. The beneficial effect of this invention is that it avoids misjudgments when relying on single prediction or judgment, and is applicable to fire pump inspection systems.

Description

Technical Field

[0001] This application relates to information processing technology, and in particular to a fire pump safety identification system and method. Background Technology

[0002] Currently, the safety assessment of fire pumps mainly relies on manual inspections, using traditional button status detection methods. This means that during manual inspections, operators must personally observe and judge the button status and dial pointer position, which carries the risk of subjective operation and misjudgment. Traditional button status detection methods typically use physical contact or sensor technology to monitor the button's on / off state, which suffers from low sensitivity, susceptibility to environmental interference, and maintenance difficulties. Summary of the Invention

[0003] The purpose of this application is to solve the problems of existing fire pump safety assessments, which require manual observation and judgment leading to misjudgment, or the low sensitivity, susceptibility to environmental interference, and maintenance difficulties when using physical contact and sensor technology. This application provides a fire pump safety identification system and method.

[0004] In the technical solution adopted by this application to solve the above-mentioned technical problems, the first aspect provides a fire pump safety identification system, including an image acquisition unit and a processing unit, wherein the image acquisition unit and the processing unit are connected.

[0005] The image acquisition unit is used to acquire video information of the inspection process by the inspection personnel and send the video information to the processing unit in real time.

[0006] The processing unit is pre-set with a start / stop operation recognition model, a dial detection model, and a pointer segmentation model. The processing unit is used to predict whether the button status of the inspected fire pump is normal based on the video information and the start / stop operation recognition model, and to predict whether the dial of the inspected fire pump is normal based on the video information, the dial detection model, and the pointer segmentation model.

[0007] Specifically, to illustrate how to predict whether the button status of the inspected fire pump is normal based on the video information and the start / stop operation recognition model, the step of predicting whether the button status of the inspected fire pump is normal based on the video information and the start / stop operation recognition model includes:

[0008] The video information is identified and divided into multiple video frames. The detection frames that contain the inspection button are extracted. Each detection frame is input into the start / stop operation recognition model in the order of the video for prediction. The ordered prediction results are used as the prediction sequence. The prediction sequence is compared with the preset sequence. If the comparison is successful, the operation is predicted to be normal; otherwise, the operation is predicted to be abnormal.

[0009] Furthermore, to illustrate how the preset start / stop operation recognition model is obtained, the method for obtaining the start / stop operation recognition model is as follows:

[0010] A first pre-trained model was established based on the image classification algorithm of ShuffleNet V2.

[0011] Acquire first raw image data for each state category of the fire pump button status indicator light. The first raw image data includes image data of the fire pump button status indicator light under different seasons and / or different time periods collected by different cameras and / or different angles. The state categories include only the indicator light corresponding to the red button is on, only the indicator light corresponding to the green button is on, both the indicator lights corresponding to the red and green buttons are on, and neither of them is on.

[0012] Data augmentation is performed on each of the first original image data to generate new first sample data;

[0013] The new first sample data and each first original image data are used together as training samples, and divided into training set and validation set according to a certain ratio to train the first pre-trained model, and the corresponding first training loss function is set.

[0014] After training, a start / stop operation recognition model is obtained.

[0015] Specifically, to provide a corresponding first training loss function, the expression of the first training loss function is as follows:

[0016] loss1=-1 / N*∑(i,j)((1-ε)*y ij *log(yhat ij )+ε*(1 / m))

[0017] Where N represents the total number of training samples, i refers to the i-th training sample, j refers to the j-th state category, and y ij For real tags, yhat ij ε is the predicted value, ε is the label smoothing coefficient, and m represents the total number of state categories.

[0018] Here, ε is generally taken as 0.1 to 0.2.

[0019] Furthermore, to illustrate how to predict whether the dial of the inspected fire pump is normal based on the video information, the dial detection model, and the pointer segmentation model, the prediction of whether the dial of the inspected fire pump is normal based on the video information, the dial detection model, and the pointer segmentation model includes:

[0020] A dial detection model is used to identify video information, resulting in multiple frames containing images of the dial of the inspected fire pump. At least one image is extracted, and the extracted image is cropped to obtain a dial image. The dial image is then input into a pointer segmentation model to obtain the current pointer angle. Based on the current pointer angle, it is determined whether the dial is functioning normally.

[0021] Specifically, to illustrate how the preset dial detection model is obtained, the method for obtaining the dial detection model is as follows:

[0022] A second pre-trained model was established based on the YOLOv5lite-s object detection algorithm.

[0023] Acquire second raw image data of the fire pump dial, the second raw image data including image data of the fire pump dial collected by different cameras and / or different angles in different seasons and / or different time periods;

[0024] Data augmentation is performed on the second original image data to generate new second sample data;

[0025] Each sample in the new second sample data and the second original image data is labeled to indicate the precise position of the fire pump dial.

[0026] Each labeled sample is used as a training sample, and the second pre-trained model is trained by dividing it into a training set and a validation set according to a certain ratio, and a corresponding second training loss function is set.

[0027] After training, a dial detection model is obtained.

[0028] Furthermore, to provide a corresponding second training loss function, the expression for the second training loss function is as follows:

[0029] loss2=λ locallizaion L locallizaion +λ class L class +λ confidence L confidence

[0030] Where, λ locallizaion The weight λ represents the position error loss. class The weights λ represent the classification error loss. confidence This represents the weight of the confidence error loss; typically, values ​​are 0.05, 0.5, and 1.0.

[0031] L locallizaion This refers to the positional error, and its formula is:

[0032]

[0033] Where IoU refers to the ratio of the intersection to the union of two bounding boxes, i.e., the Intersection over Union (IoU); c is the Euclidean distance between the center points of the detection box and the ground truth box; ρ is the difference between the aspect ratios of the detection box and the ground truth box; and b represents the position of the detection box. gt This indicates the position of the true bounding box, and α represents the balance coefficient, which can be set as needed to adjust the importance between different indicators.

[0034]

[0035] Here, w refers to the width of the detection box, and h refers to the height of the detection box. gt This refers to the width of the actual bounding box, h. gt This refers to the actual height of the bounding box;

[0036] L confidence This refers to the confidence error, and its formula is:

[0037] L confidence = -t*log(p) - (1-t)*log(1-p)

[0038] Where t is 1 or 0, indicating whether the current detection box contains an object, and p represents the probability that an object exists;

[0039] L class This refers to classification error, and its formula is:

[0040] L class =-logqr

[0041] q refers to the probability that the second pre-trained model predicts whether the current sample is a watch face image, and r refers to whether the current sample is actually a watch face image.

[0042] Specifically, to illustrate how the preset pointer segmentation model is obtained, the method for obtaining the pointer segmentation model is as follows:

[0043] A third pre-trained model is established based on the UNet object segmentation algorithm.

[0044] Acquire third raw image data of the fire pump dial pointer, the third raw image data including image data of the fire pump dial pointer collected by different cameras and / or different angles in different seasons and / or different time periods;

[0045] Data augmentation is performed on the third original image data to generate new third sample data;

[0046] The samples in the new third sample data and the third original image data are segmented and labeled to indicate the position of the fire pump dial pointer;

[0047] Each sample after segmentation and labeling is used as a training sample. The sample is divided into a training set and a validation set according to a certain ratio to train the third pre-trained model, and a corresponding third training loss function is set.

[0048] After training, the pointer segmentation model is obtained.

[0049] Furthermore, to provide a corresponding third training loss function, the expression for the third training loss function is as follows:

[0050]

[0051] Where H is the number of current training samples, x d 'a' represents the true category, which is the category manually labeled in the training samples, and 'a' represents the predicted category. The categories in the true category and the predicted category include the background category and the pointer category.

[0052] Specifically, to alert inspection personnel and / or other staff, a notification unit is also included, which is connected to the processing unit;

[0053] The prompting unit is used to issue corresponding prompts based on the prediction results of the processing unit.

[0054] In the technical solution adopted by this application to solve the above-mentioned technical problems, the second aspect provides a method for safety identification of fire pumps, including the following steps:

[0055] Upon arrival at the fire pump to be inspected, the inspection begins, and the fire pump buttons are operated in the pre-agreed sequence.

[0056] The video information of the inspection process by the inspection personnel is collected by the image acquisition unit of the fire pump safety identification system described above, and the video information is sent to the processing unit in real time.

[0057] The processing unit predicts whether the button status of the inspected fire pump is normal based on the video information and the start / stop operation recognition model, and predicts whether the dial of the inspected fire pump is normal based on the video information, the dial detection model, and the pointer segmentation model.

[0058] The beneficial effect of this application is that, in the solution of this application, the button status and dial pointer of the fire pump being inspected during the inspection process are identified and predicted by the fire pump safety identification system, which reduces the misjudgment when making a manual judgment, and can be combined with manual judgment to provide secondary verification for the inspection personnel, avoiding misjudgment when making a single prediction or judgment. Attached Figure Description

[0059] Figure 1 This is a system block diagram of the fire pump safety identification system provided in the first aspect of the embodiments of this application.

[0060] Figure 2 This is a schematic flowchart of a fire pump safety identification method provided in the second aspect of the embodiments of this application. Detailed Implementation

[0061] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0062] It should be noted that when a component is referred to as being "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as being "connected to" another component, it can be directly connected to or indirectly connected to that other component.

[0063] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0064] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0065] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0066] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0067] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0068] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0069] Figure 1 A system block diagram of a fire pump safety identification system provided in the first aspect of the present application is shown. For ease of explanation, only the parts related to this embodiment are shown.

[0070] The fire pump safety identification system includes an image acquisition unit and a processing unit, wherein the image acquisition unit and the processing unit are connected.

[0071] Here, the image acquisition unit is used to acquire video information of the inspection process by the inspection personnel and send the video information to the processing unit in real time;

[0072] The processing unit is pre-set with a start / stop operation recognition model, a dial detection model, and a pointer segmentation model. The processing unit is used to predict whether the button status of the inspected fire pump is normal based on the received video information and the start / stop operation recognition model, and to predict whether the dial of the inspected fire pump is normal based on the received video information, the dial detection model, and the pointer segmentation model.

[0073] Understandably, the aforementioned fire pump safety identification system uses image processing to identify fire pumps for safety purposes. This system does not require physical contact with the inspected fire pumps, nor does it use sensors, thus avoiding problems such as low sensitivity, susceptibility to environmental interference, and maintenance difficulties. Furthermore, the system's safety identification of fire pumps avoids misjudgments that can occur with manual prediction or assessment alone.

[0074] Furthermore, the image acquisition unit here can be a standalone device consisting of a camera and a wireless communication module. It interacts with the processing unit via the wireless communication module and can be made into a wearable device for inspection personnel, avoiding the inconvenience of handheld operation. Alternatively, the image acquisition unit can be a smartphone or other mobile communication terminal with a camera function—any device capable of acquiring video information and interacting with the processing unit. The purpose of using wireless communication is to place the processing unit on a server, offloading most of the computationally intensive processing and / or recognition functions remotely. This separates the image acquisition unit from the processing unit, reducing the size of the device worn or held by inspection personnel and saving costs.

[0075] Of course, the image acquisition unit and the processing unit can also exist in one device or equipment, but this would be relatively expensive.

[0076] In some embodiments, to illustrate how to predict whether the button status of the inspected fire pump is normal based on video information and the start / stop operation recognition model, predicting whether the button status of the inspected fire pump is normal based on the received video information and the start / stop operation recognition model may include:

[0077] The video information is identified and divided into multiple video frames. The detection frames that contain the inspection button are extracted. Each detection frame is input into the start / stop operation recognition model in the order of the video for prediction. The ordered prediction results are used as the prediction sequence. The prediction sequence is compared with the preset sequence. If the comparison is successful, the operation is predicted to be normal; otherwise, the operation is predicted to be abnormal.

[0078] Understandably, the preset sequence should correspond to the pre-agreed operating order of the fire pump buttons by the inspection personnel. For example, setting only the indicator light corresponding to the red button as 1, only the indicator light corresponding to the green button as 2, both red and green buttons as 3, and neither red nor green button as 0, the preset sequence could be 0, 1, 3. Then, a predicted sequence of 0, 2, 3 indicates an operational anomaly, while a predicted sequence of 0, 1, 3 indicates normal operation.

[0079] In some embodiments, to illustrate how the preset start / stop operation recognition model is obtained, the method for obtaining the start / stop operation recognition model may be as follows:

[0080] A first pre-trained model was established based on the image classification algorithm of ShuffleNet V2.

[0081] Acquire first raw image data of the fire pump button status indicator under each status category. The first raw image data includes image data of the fire pump button status indicator under each status category collected by different cameras and / or different angles in different seasons (spring, summer, autumn, winter) and / or different time periods (e.g., every hour between 8:00 and 18:00 is considered as a time period, which is the time for general inspection). Here, the status categories include only the indicator light corresponding to the red button is on, only the indicator light corresponding to the green button is on, both the red and green button indicator lights are on, and neither is on.

[0082] Data augmentation is performed on each of the first original image data to generate new first sample data;

[0083] The new first sample data and each first original image data are used together as training samples, and divided into training set and validation set according to a certain ratio to train the first pre-trained model, and the corresponding first training loss function is set.

[0084] After training, a start / stop operation recognition model is obtained.

[0085] Understandably, data augmentation here can refer to techniques such as flipping and / or rotating and / or cropping and / or deforming and / or scaling the original image data, with the aim of ensuring the generalization ability of the trained model. A suitable ratio here could be 8:2.

[0086] In some embodiments, to provide a corresponding first training loss function, the expression of the first training loss function can be:

[0087] loss1=-1 / N*∑(i,j)((1-ε)*y ij *log(yhat ij )+ε*(1 / m))

[0088] Where N represents the total number of training samples, i refers to the i-th training sample, j refers to the j-th state category, and y ij For real tags, yhat ij ε is the predicted value, ε is the label smoothing coefficient, and m represents the total number of state categories.

[0089] Here, ε is generally taken as 0.1 to 0.2.

[0090] Understandably, the label smoothing term adds a mean (1 / m) correction term to the probability of each state category. During training, the gradient is obtained through backpropagation, thus updating the network parameters. The purpose of label smoothing is to alleviate overfitting in the trained model, reduce classification errors, and improve generalization ability. In the original cross-entropy loss function, when the probability of one category is 1, the probabilities of other categories become 0, which can easily lead to edge errors in the network, thus affecting the overall performance of the trained model. However, when using the label smoothing term, the probability distribution of the category labels is forced to be smoothed, resulting in a more uniform distribution of label probabilities, thereby reducing edge errors.

[0091] In some embodiments, to illustrate how to predict whether the dial of the inspected fire pump is normal based on the received video information, the dial detection model, and the pointer segmentation model, the prediction of whether the dial of the inspected fire pump is normal based on the received video information, the dial detection model, and the pointer segmentation model may include:

[0092] A dial detection model is used to identify video information, resulting in multiple frames containing images of the dial of the inspected fire pump. At least one image is extracted, and the extracted image is cropped to obtain the dial image. The dial image is then input into a pointer segmentation model to obtain the current pointer angle. Based on the current pointer angle, it is determined whether the dial is functioning normally.

[0093] Understandably, under normal circumstances, the current dial is considered abnormal only if the current pointer angle is 0.

[0094] In some embodiments, to illustrate how the preset dial detection model is obtained, the method for obtaining the dial detection model may be as follows:

[0095] A second pre-trained model was established based on the YOLOv5lite-s object detection algorithm.

[0096] Acquire second raw image data of the fire pump dial, which includes image data of the fire pump dial collected by different cameras and / or different angles in different seasons and / or different time periods;

[0097] Data augmentation is performed on the second original image data to generate new second sample data;

[0098] Each sample in the new second sample data and the second original image data is labeled to indicate the precise position of the fire pump dial.

[0099] Each labeled sample is used as a training sample, and the second pre-trained model is trained by dividing it into a training set and a validation set according to a certain ratio, and a corresponding second training loss function is set.

[0100] After training, a dial detection model is obtained.

[0101] Understandably, data augmentation here can also refer to data augmentation techniques such as flipping and / or rotating and / or cropping and / or deforming and / or scaling the second original image data, with the aim of ensuring the generalization ability of the trained model. The ratio here can also be 8:2.

[0102] In some embodiments, to provide a corresponding second training loss function, the expression for the second training loss function may be:

[0103] loss2=λ locallizaion L locallizaion +λ class L class +λ confidence L confidence

[0104] Where, λ locallizaion The weight λ represents the position error loss. class The weights λ represent the classification error loss. confidence This represents the weight of the confidence error loss; typically, values ​​are 0.05, 0.5, and 1.0.

[0105] L locallizaion This refers to the positional error, and its formula is:

[0106]

[0107] Where IoU (Intersection over Union) refers to the ratio of the intersection to the union of two bounding boxes, i.e., the Intersection over Union ratio; c (center) is the Euclidean distance between the center points of the detection box and the ground truth box, used to measure positional similarity, and its range is [0, +∞); ρ (aspect ratio) is the difference between the aspect ratios of the detection box and the ground truth box, used to measure aspect ratio similarity, and its range is [0, +∞); b represents the position of the detection box, b... gt This indicates the position of the true bounding box, and α represents the balance coefficient, which can be set as needed to adjust the importance between different indicators. α and v together are called the influence factor.

[0108]

[0109] Here, w refers to the width of the detection box, and h refers to the height of the detection box. gt This refers to the width of the actual bounding box, h. gt This refers to the actual height of the bounding box;

[0110] L confidence This refers to the confidence error, and its formula is:

[0111] L confidence = -t*log(p) - (1-t)*log(1-p)

[0112] Where t is 1 or 0, indicating whether the current detection box contains an object, and p represents the probability that an object exists;

[0113] L class This refers to classification error, and its formula is:

[0114] L class =-logqr

[0115] q refers to the probability that the second pre-trained model predicts whether the current sample is a watch face image, and r refers to whether the current sample is actually a watch face image, usually represented by one-hot encoding.

[0116] In some embodiments, to illustrate how the preset pointer segmentation model is obtained, the method for obtaining the pointer segmentation model may be as follows:

[0117] A third pre-trained model is established based on the UNet object segmentation algorithm.

[0118] Acquire third raw image data of the fire pump dial pointer, which includes image data of the fire pump dial pointer collected by different cameras and / or different angles in different seasons and / or different time periods;

[0119] Data augmentation is performed on the third original image data to generate new third sample data;

[0120] The samples in the new third sample data and the third original image data are segmented and labeled to indicate the position of the fire pump dial pointer;

[0121] Each sample after segmentation and labeling is used as a training sample. The sample is divided into a training set and a validation set according to a certain ratio to train the third pre-trained model, and a corresponding third training loss function is set.

[0122] After training, the pointer segmentation model is obtained.

[0123] Understandably, data augmentation here can also refer to data augmentation techniques such as flipping and / or rotating and / or cropping and / or deforming and / or scaling of the third original image data, with the aim of ensuring the generalization ability of the trained model. The ratio here can also be 8:2.

[0124] In some embodiments, to provide a corresponding third training loss function, the expression of the third training loss function can be:

[0125]

[0126] Where H is the number of current training samples, x d Let represent the true category, i.e., the category manually labeled in the training samples, and 'a' represent the predicted category. Here, the categories in the true and predicted categories include the background category and the pointer category. Since hardware computing resources are limited during neural network training, a large dataset (i.e., a set of samples) is sometimes divided into several smaller batches. For example, if 100 samples are divided into 5 batches on average, then H is 20.

[0127] In some embodiments, a notification unit may be included to alert inspection personnel and / or other staff, and this notification unit is connected to the processing unit.

[0128] The prompting unit is used to issue corresponding prompts based on the prediction results of the processing unit.

[0129] It is understandable that the connection between the prompting unit and the processing unit can be either wired or wireless.

[0130] In addition, the prompting unit can be a buzzer and / or an indicator light and / or a remote communication device. When it is a remote communication device, if it receives a prompting request sent by the processing unit, it can remotely notify the mobile communication terminals such as mobile phones of the inspection personnel and / or management personnel. When the prompting unit is a buzzer and / or an indicator light, it can be used together with the image acquisition unit as a separate device, worn or held by the inspection personnel to prompt them, or it can be placed near the processing unit to prompt the management personnel.

[0131] Figure 2 A schematic flowchart illustrating a fire pump safety identification method provided in the second aspect of an embodiment of this application is shown. This is by way of example and not limitation. The fire pump safety identification method includes:

[0132] S101. Arrive at the fire pump to be inspected and begin the inspection. Operate the fire pump buttons in the pre-agreed sequence.

[0133] S102. The video information of the inspection process of the inspection personnel is collected by the image acquisition unit of the fire pump safety identification system as described above, and the video information is sent to the processing unit in real time.

[0134] S103. The processing unit predicts whether the button status of the inspected fire pump is normal based on the received video information and the start / stop operation recognition model, and predicts whether the dial of the inspected fire pump is normal based on the received video information, the dial detection model and the pointer segmentation model.

[0135] It is understandable that the pre-agreed order here refers to the order of fire pump button operation agreed upon with the inspection personnel in the above scheme, which corresponds to the preset sequence.

[0136] In addition, to provide real-time alerts to inspection personnel and / or management personnel, when the aforementioned fire pump safety identification system has an alert unit, the processing unit can immediately notify or control the alert unit to issue a corresponding early warning when it predicts that the button status of the inspected fire pump is abnormal, and immediately notify or control the alert unit to issue a corresponding early warning when it predicts that the dial is abnormal.

[0137] In some embodiments, in order to save computing resources, the processing unit may, when it predicts that the button status of the inspected fire pump is abnormal, notify or control the prompting unit to issue a corresponding early warning prompt, and then stop predicting whether the dial is normal, thereby saving subsequent computing resources.

[0138] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0139] It should be noted that the information interaction and execution process between the above-mentioned devices / units / modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0142] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0143] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

[0147] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A fire pump safety identification system, characterized in that, It includes an image acquisition unit and a processing unit, wherein the image acquisition unit and the processing unit are connected; The image acquisition unit is used to acquire video information of the inspection process by the inspection personnel and send the video information to the processing unit in real time. The processing unit is pre-set with a start / stop operation recognition model, a dial detection model, and a pointer segmentation model. The processing unit is used to predict whether the button status of the inspected fire pump is normal based on the video information and the start / stop operation recognition model, and to predict whether the dial of the inspected fire pump is normal based on the video information, the dial detection model, and the pointer segmentation model. The step of predicting whether the button status of the inspected fire pump is normal based on the video information and the start / stop operation recognition model includes: The video information is identified and divided into multiple video frames. The detection frames that contain the inspection button detection are extracted. Each detection frame is input into the start / stop operation recognition model in the order of the video for prediction. The ordered prediction results are obtained as the prediction sequence. The prediction sequence is compared with the preset sequence. If the comparison is successful, the operation is predicted to be normal; otherwise, the operation is predicted to be abnormal. The method for obtaining the start / stop operation recognition model is as follows: A first pre-trained model was established based on the image classification algorithm of ShuffleNet V2. Acquire first raw image data for each state category of the fire pump button status indicator light. The first raw image data includes image data of the fire pump button status indicator light under different seasons and / or different time periods collected by different cameras and / or different angles. The state categories include only the indicator light corresponding to the red button is on, only the indicator light corresponding to the green button is on, both the indicator lights corresponding to the red and green buttons are on, and neither of them is on. Data augmentation is performed on each of the first original image data to generate new first sample data; The new first sample data and each first original image data are used together as training samples, and divided into training set and validation set according to a certain ratio to train the first pre-trained model, and the corresponding first training loss function is set. After training, a start / stop operation recognition model is obtained; When the processing unit predicts that the button status of the inspected fire pump is abnormal, it will notify or control the prompting unit to issue a corresponding early warning, and then will no longer predict whether the dial is normal.

2. The fire pump safety identification system as described in claim 1, characterized in that, The expression for the first training loss function is: loss1=-1 / N*∑(i,j)((1-ε)*y ij *log(yhat ij )+ε*(1 / m)) Where N represents the total number of training samples, i refers to the i-th training sample, j refers to the j-th state category, and y ij For real tags, yhat ij ε is the predicted value, ε is the label smoothing coefficient, and m represents the total number of state categories.

3. The fire pump safety identification system as described in claim 1, characterized in that, The step of predicting whether the dial of the inspected fire pump is normal based on the video information, the dial detection model, and the pointer segmentation model includes: A dial detection model is used to identify video information, resulting in multiple frames containing images of the dial of the inspected fire pump. At least one image is extracted, and the extracted image is cropped to obtain a dial image. The dial image is then input into a pointer segmentation model to obtain the current pointer angle. Based on the current pointer angle, it is determined whether the dial is functioning normally.

4. The fire pump safety identification system as described in claim 1, characterized in that, The method for obtaining the dial detection model is as follows: A second pre-trained model was established based on the YOLOv5lite-s object detection algorithm. Acquire second raw image data of the fire pump dial, the second raw image data including image data of the fire pump dial collected by different cameras and / or different angles in different seasons and / or different time periods; Data augmentation is performed on the second original image data to generate new second sample data; Each sample in the new second sample data and the second original image data is labeled to indicate the precise position of the fire pump dial. Each labeled sample is used as a training sample, and the second pre-trained model is trained by dividing it into a training set and a validation set according to a certain ratio, and a corresponding second training loss function is set. After training, a dial detection model is obtained.

5. The fire pump safety identification system as described in claim 4, characterized in that, The expression for the second training loss function is: loss2=λ locallizaion L locallizaion +λ class L class +λ confidence L confidence Where, λ locallizaion The weight λ represents the position error loss. class The weights λ represent the classification error loss. confidence The weight representing the confidence error loss; L locallizaion This refers to the positional error, and its formula is: Where IoU refers to the ratio of the intersection to the union of two bounding boxes, i.e., the Intersection over Union (IoU); c is the Euclidean distance between the center points of the detection box and the ground truth box; ρ is the difference between the aspect ratios of the detection box and the ground truth box; and b represents the position of the detection box. gt This indicates the position of the true bounding box, and α represents the balance coefficient. Here, w refers to the width of the detection box, and h refers to the height of the detection box. gt This refers to the width of the actual bounding box, h. gt This refers to the actual height of the bounding box; L confidence This refers to the confidence error, and its formula is: L confidence =-t*log(p)-(1-t)*log(1-p) Where t is 1 or 0, indicating whether the current detection box contains an object, and p represents the probability that an object exists; L class This refers to classification error, and its formula is: L class =-logqr q refers to the probability that the second pre-trained model predicts whether the current sample is a watch face image, and r refers to whether the current sample is actually a watch face image.

6. The fire pump safety identification system as described in any one of claims 1-5, characterized in that, The method for obtaining the pointer segmentation model is as follows: A third pre-trained model is established based on the UNet object segmentation algorithm. Acquire third raw image data of the fire pump dial pointer, the third raw image data including image data of the fire pump dial pointer collected by different cameras and / or different angles in different seasons and / or different time periods; Data augmentation is performed on the third original image data to generate new third sample data; The samples in the new third sample data and the third original image data are segmented and labeled to indicate the position of the fire pump dial pointer; Each sample after segmentation and labeling is used as a training sample. The sample is divided into a training set and a validation set according to a certain ratio to train the third pre-trained model, and a corresponding third training loss function is set. After training, the pointer segmentation model is obtained.

7. The fire pump safety identification system as described in claim 6, characterized in that, The expression for the third training loss function is: Where H is the number of current training samples, x d 'a' represents the true category, which is the category manually labeled in the training samples, and 'a' represents the predicted category. The categories in the true category and the predicted category include the background category and the pointer category.

8. A method for safety identification of fire pumps, characterized in that, Includes the following steps: Upon arrival at the fire pump to be inspected, the inspection begins, and the fire pump buttons are operated in the pre-agreed sequence. The video information of the inspection process of the inspection personnel is collected by the image acquisition unit of the fire pump safety identification system as described in any one of claims 1-7, and the video information is sent to the processing unit in real time. The processing unit predicts whether the button status of the inspected fire pump is normal based on the video information and the start / stop operation recognition model, and predicts whether the dial of the inspected fire pump is normal based on the video information, the dial detection model, and the pointer segmentation model.

Images