Method and system for judging fire degree of electric bicycle based on image recognition

Abstract

The invention discloses an electric bicycle fire hazard degree determination method and system based on image identification. The electric bicycle fire hazard degree determination method based on image identification comprises the following steps: obtaining a monitoring video stream of an electric bicycle parking point; intercepting a real-time image from the monitoring video stream, adding the real-time image into a backtracking image set according to a preset time interval, identifying the real-time image by using a flame smoke identification model, judging whether a fire occurrence condition is met or not, if the fire occurrence condition is not met, continuing monitoring, and if the fire occurrence condition is met, stopping monitoring. If yes, determining an optimal backtracking image from the backtracking image set; and acquiring a fire risk parameter in the optimal backtracking image, and determining the fire hazard degree of the electric bicycle according to the fire risk parameter and a preset fire related parameter. The electric bicycle fire hazard risk can be quantified so that the accuracy of fire hazard danger degree determination can be enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of fire monitoring, and in particular to a method and system for judging the fire degree of an electric bicycle based on image recognition. Background Art

[0002] With the rapid growth of the electric bicycle market in China, they have become an important part of urban transportation, especially favored by office workers, students and housewives. However, the accompanying problem of electric bicycle fires has become increasingly prominent, not only threatening people's lives and safety, but also causing a lot of economic losses. Existing electric bicycle fire prevention and control technologies are mainly concentrated in the fire warning stage, such as monitoring the status of the charging device and using smoke and flame detectors to identify the initial fire. Although these measures can help to warn of fires in advance, they cannot automatically assess the danger level of the fire according to the specific circumstances of the fire. Therefore, when facing a fire, rescuers must rely on personal experience and intuition to judge the severity of the fire. In addition, electric bicycle fires are characterized by dense smoke and bright flames. After the fire, the on-site monitoring screen is greatly affected, affecting the performance of manual judgment and artificial intelligence image recognition related algorithms, which directly affects the effectiveness and timeliness of the rescue strategy. Summary of the invention

[0003] The main purpose of the present invention is to provide a method and system for judging the fire degree of an electric bicycle based on image recognition, aiming to solve the technical problem of improving the accuracy of judging the fire hazard degree of an electric bicycle in the background technology.

[0004] To achieve the above object, the present invention provides a method for determining the degree of fire of an electric bicycle based on image recognition, which is characterized by comprising the following steps:

[0005] Step S100: Acquire a surveillance video stream of an electric bicycle parking spot;

[0006] Step S200: intercepting a real-time image from the monitoring video stream, adding the real-time image to the retrospective image set according to a preset time interval, using a flame and smoke recognition model to recognize the real-time image, and judging whether the fire occurrence condition is met. If the fire occurrence condition is met, executing step S300; if the fire occurrence condition is not met, continuing monitoring;

[0007] Step S300: determining the best retrospective image from the retrospective image set;

[0008] Step S400: Obtain fire risk parameters in the optimal retrospective image, and determine the fire hazard level of the electric bicycle according to the fire risk parameters and preset fire-related parameters.

[0009] Furthermore, in step S300, the step of determining the best retrospective image from the retrospective image set includes:

[0010] Step S310: obtaining the most recent k retrospective images from the retrospective image set, where k is a preset positive integer;

[0011] Step S320: using the electric bicycle recognition analysis model to recognize the k retrospective images, and calculating the number of electric bicycles in each image;

[0012] Step S330: using the combustible material recognition model to recognize the k retrospective images, and calculating the number of combustible materials in each image;

[0013] Step S340: determining the best retrospective image according to the number, density and amount of combustibles of the k retrospective images.

[0014] Furthermore, in step S340, determining the best retrospective image according to the number, density and quantity of combustibles of the k retrospective images includes the following steps:

[0015] Step S341: scoring the k retrospective images, the scoring formula is:

[0016]

[0017] Among them, n represents the number of electric bicycles, m represents the number of combustibles, t represents the time of backtracking the current distance, F is the sensitivity factor, which is used to adjust the influence of the number of electric bicycles and the number of combustibles, and G is the gravity factor, which is used to adjust the influence of time;

[0018] Step S342: Select the retrospective image with the highest score as the best retrospective image.

[0019] Furthermore, in step S400, the step of obtaining the fire risk parameters in the optimal retrospective image and determining the fire risk level of the electric bicycle according to the fire risk parameters and preset fire-related parameters includes:

[0020] Step S410: obtaining preset fire-related parameters, wherein the preset fire-related parameters include the number of electric bicycles designed to be parked and a disaster consequence score;

[0021] Step S420: determining the number score of electric bicycles according to the number of electric bicycles in the best backtracking image;

[0022] Step S430: calculating the density of electric bicycles according to the number of electric bicycles in the optimal backtracking image and the designed parking number of electric bicycles, and determining the overload coefficient score according to the density of electric bicycles;

[0023] Step S440: Calculate the fire risk value of electric bicycles according to the number score of electric bicycles, the overload coefficient score and the disaster consequence score, and output the corresponding danger level according to the fire risk value of the electric bicycles.

[0024] Furthermore, the step S400 further includes the following steps:

[0025] Step S500: Regularly check the online update conditions of the model according to a preset statistical period, and perform online update or continue monitoring according to the check result.

[0026] Furthermore, in step S500, the online update condition of the model is regularly checked according to a preset statistical period, and the step of performing online update or continuing monitoring according to the check result includes:

[0027] Step S510: Count the prediction performance of the model according to a preset statistical period, and store the statistical results in a statistical database;

[0028] Step S520: Check whether the model meets one of the following online update conditions: the model online time reaches the preset update cycle threshold, the model performance index is lower than the preset threshold, and the prediction accuracy value of the historical N statistical cycles including the current statistical cycle continues to decrease. If any of the conditions is met, execute step S530. If none of them are met, continue to monitor the model status and wait for the next periodic check;

[0029] Step S530: Acquire the latest image as training data and perform online training on the model to obtain an updated model.

[0030] Further, in step S320, the electric bicycle recognition analysis model is trained by an improved loss function, specifically:

[0031] L=L Attr +α*L RepGT +β*L RepBox +γ*L num

[0032] Among them, L is the total loss function, which is used to comprehensively evaluate the prediction performance of the model; L Attr is the attraction term, which is used to make the predicted box closer to the real box; L RepGT To exclude the real box item, it is used to make the predicted box away from the surrounding real box; L RepBox To exclude the prediction box item, it is used to keep the prediction box away from other surrounding prediction boxes; L num is the quantity loss term, which is used to make the number of predicted boxes closer to the number of real boxes; parameters α, β and γ are used to balance L RepGT , L RepBox , Lnum The weight of .

[0033] Furthermore, in step S320, each item in the loss function of the electric bicycle identification and analysis model is specifically defined as follows:

[0034]

[0035] Among them, Smooth L1 is a smooth L1 loss function used to calculate the regression loss between the predicted box and the true box; P is the predicted box; B P It is the bounding box adjusted by regression based on the predicted box P; It is the real box with the maximum IoU intersection and union ratio with the predicted box P, that is is the size of the positive sample prediction box set, that is, the number of prediction boxes in the set;

[0036]

[0037] Among them, Smooth ln It is a smooth logarithmic loss function used to calculate the distance between the predicted box and the surrounding real box; is the other real box with the maximum IoU with the predicted box P, that is and For the intersection-over-union ratio, calculate the predicted box B P With real box The overlapping area and the ground-truth box The ratio of the total area,

[0038]

[0039] Among them, L RepBox To exclude the predicted box items, the smooth logarithmic loss function Smooth is used ln ;∈ is a small constant used to prevent the denominator from being zero; P i and P j for Different prediction boxes in the positive sample prediction box set, and is the predicted box P i and P j The bounding box of is the indicator function, when Returns 1 if true, otherwise returns 0.

[0040]

[0041] Among them, L numis the quantity loss term, and num is the absolute value of the difference between the predicted number of electric bicycles in the image and the actual number of electric bicycles.

[0042] The second aspect of the present invention provides an electric bicycle fire severity determination system based on image recognition, comprising:

[0043] A collection module is used to obtain the monitoring video stream of the electric bicycle parking point;

[0044] an identification module, for capturing a real-time image from the monitoring video stream, adding the real-time image to a retrospective image set according to a preset time interval, identifying the real-time image using a flame and smoke identification model, and determining whether a fire occurrence condition is met; if the fire occurrence condition is met, the control determination module determines the best retrospective image from the retrospective image set; if the fire occurrence condition is not met, the monitoring is continued;

[0045] A determination module, used to determine the best retrospective image from the retrospective image set;

[0046] A judgment module, used to obtain the fire risk parameters in the optimal retrospective image, and determine the fire hazard level of the electric bicycle according to the fire risk parameters and preset fire-related parameters;

[0047] The update module is used to regularly check the online update conditions of the model according to the preset statistical period, and perform online updates or continue monitoring based on the inspection results.

[0048] Furthermore, the electric bicycle fire severity determination system based on image recognition also includes:

[0049] The update module is used to regularly check the online update conditions of the water pump group health status monitoring model according to a preset statistical period, and perform online update or continue monitoring according to the inspection results.

[0050] The third aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for determining the degree of fire of an electric bicycle based on image recognition as described in the first aspect when executing the computer program.

[0051] The fourth aspect of the present invention provides a computer-readable storage medium, on which is stored an electric bicycle fire severity judgment processing program based on image recognition. When the electric bicycle fire severity judgment processing program based on image recognition is executed by a processor, the steps of the electric bicycle fire severity judgment method based on image recognition as described in the first aspect are implemented.

[0052] Compared with the prior art, the beneficial effects of the present invention include at least:

[0053] The present invention provides a method and system for judging the extent of fire of electric bicycles based on image recognition, wherein the method for judging the extent of fire of electric bicycles based on image recognition can more accurately identify the conditions for fire occurrence through the combined use of a flame smoke recognition model, an electric bicycle recognition analysis model and a combustible material recognition model, the flame smoke recognition model can effectively detect the smoke or flame generated by the initial fire, and the electric bicycle recognition analysis model can accurately locate the position and number of electric bicycles that may cause the fire; when the system detects a possible fire signal, it will select the best retrospective image that best reflects the fire condition from a series of images taken recently, and this process is achieved by calculating the image score, and the scoring criteria take into account factors such as the number, density and number of combustible materials of electric bicycles, so as to provide key information to decision makers in the shortest time and speed up the preparation and implementation of rescue. The present invention not only issues an alarm after discovering a fire, but more importantly, it can calculate a more accurate fire risk value based on information such as the number, density and number of combustible materials of electric bicycles, combined with preset fire-related parameters, and this risk value can be used to guide the rescue team to formulate a more reasonable response strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without creative work. The purpose, functional features and advantages of the present invention will be further explained in combination with the embodiments and with reference to the drawings.

[0055] Figure 1 A flow chart of a method for determining the degree of fire in an electric bicycle based on image recognition provided by an embodiment of the present invention;

[0056] Figure 2 A flow chart of a method for determining an optimal retrospective image provided by an embodiment of the present invention;

[0057] Figure 3 A scoring flow chart for determining the best retrospective image provided by an embodiment of the present invention;

[0058] Figure 4 A flow chart of a method for calculating the fire risk value of an electric bicycle provided in an embodiment of the present invention;

[0059] Figure 5 A flowchart of another method for determining the degree of fire of an electric bicycle based on image recognition provided by an embodiment of the present invention;

[0060] Figure 6 A flow chart of a method for periodically checking whether a model meets online update conditions according to a preset statistical period provided by an embodiment of the present invention;

[0061] Figure 7 A schematic diagram of the architecture of an electric bicycle fire severity determination model based on image recognition provided by an embodiment of the present invention;

[0062] Figure 8 A schematic diagram of generating a random pixel mask within a marked box provided in an embodiment of the present invention;

[0063] Fig. 9 A schematic diagram of cutting and assembling multiple flame image annotation frames provided in an embodiment of the present invention;

[0064] Fig.10 A schematic diagram of a system for determining the degree of fire of an electric bicycle based on image recognition provided by an embodiment of the present invention;

[0065] Fig.11 A schematic diagram of a computer device for judging the extent of fire in an electric bicycle based on image recognition provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0068] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing in the full text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0069] In view of this, the main purpose of the present invention is to provide a method and system for judging the fire degree of electric bicycles based on image recognition, aiming to solve the technical problem of improving the accuracy of judging the fire hazard degree of electric bicycles in the background technology.

[0070] To achieve the above object, the present invention provides a method for determining the degree of fire of an electric bicycle based on image recognition, which is characterized by comprising the following steps:

[0071] Step S100: Acquire a surveillance video stream of an electric bicycle parking spot;

[0072] Step S200: intercepting a real-time image from the monitoring video stream, adding the real-time image to the retrospective image set according to a preset time interval, using a flame and smoke recognition model to recognize the real-time image, and judging whether the fire occurrence condition is met. If the fire occurrence condition is met, executing step S300; if the fire occurrence condition is not met, continuing monitoring;

[0073] Step S300: determining the best retrospective image from the retrospective image set;

[0074] Step S400: Obtain fire risk parameters in the optimal retrospective image, and determine the fire hazard level of the electric bicycle according to the fire risk parameters and preset fire-related parameters.

[0075] Specifically, refer to Figure 1-Figure 6The present invention obtains the monitoring video stream in real time through the high-definition camera installed at the parking point of the electric bicycle, and transmits the video stream to the central server for storage. A real-time image is intercepted from the monitoring video stream every second (which can be set according to the performance of the model and the equipment), and a real-time image is added to the retrospective image set according to a preset time interval (for example, every 20 minutes). Each real-time image is identified using a pre-trained flame and smoke recognition model to determine whether there is flame or smoke. If no flame or smoke is detected and the fire occurrence condition is not met, the monitoring continues; if flame or smoke is detected and the fire occurrence condition is met, the best retrospective image is determined from the retrospective image set, and each image in the retrospective image set is analyzed to select the image that best reflects the fire risk as the best retrospective image. Fire risk parameters are extracted from the best retrospective image, including the number, density and number of combustibles of electric bicycles, etc., and the electric bicycle fire risk value NOC is calculated according to the extracted fire risk parameters and preset fire-related parameters (such as the number of electric bicycles designed to be parked and the disaster consequence scoring standard), and the corresponding danger level is output according to the risk value NOC.

[0076] It is understandable that by capturing real-time images in the video for flame and smoke recognition, the system can detect potential fire risks in a timely manner and improve the timeliness and effectiveness of fire warnings. By comprehensively considering factors such as the number, density, and amount of combustibles of electric bicycles, the system can comprehensively assess fire risks, provide more scientific judgments on the degree of fire danger, and help take timely and effective response measures. An image is saved to the retrospective image set every 20 minutes to ensure that key historical images are retained even during long-term monitoring, which is convenient for subsequent analysis and tracing.

[0077] Furthermore, in step S300, the step of determining the best retrospective image from the retrospective image set includes:

[0078] Step S310: obtaining the most recent k retrospective images from the retrospective image set, where k is a preset positive integer;

[0079] Step S320: using the electric bicycle recognition analysis model to recognize the k retrospective images, and calculating the number of electric bicycles in each image;

[0080] Step S330: using the combustible material recognition model to recognize the k retrospective images, and calculating the number of combustible materials in each image;

[0081] Step S340: determining the best retrospective image according to the number, density and amount of combustibles of the k retrospective images.

[0082] Specifically, the most recent k retrospective images are obtained from the retrospective image set, where k is a preset positive integer (eg, k=15). image=image1,image2,...,image k , image1 is the most recent image. Use the electric bicycle recognition analysis model to identify the k retrospective images and calculate the number of electric bicycles in each image, recorded as N bike =n1,n2,......,n k , n k For image k The combustibles recognition model is used to identify the k traceback images, and the number of combustibles in each image is calculated, which is recorded as Combustibles = m1, m2, ..., m k , m k For image k The number of combustibles in the image, including clothes, cartons, and awnings. Then, the best retrospective image is determined based on the number of electric bicycles and combustibles in the k retrospective images. By using the electric bicycle recognition analysis model to calculate the number of electric bicycles in each image, the fire risk can be evaluated more comprehensively. The more electric bicycles there are, the higher the fire risk. By using the combustible recognition model to calculate the number of combustibles in each image, the potential hazards of the fire can be evaluated more accurately. The more combustibles there are, the greater the possibility of the fire spreading. By obtaining the most recent k images from the retrospective image set and selecting the best retrospective image from them, it can be ensured that the selected image is timely and can reflect the latest fire situation.

[0083] Furthermore, in step S340, determining the best retrospective image according to the number, density and quantity of combustibles of the k retrospective images includes the following steps:

[0084] Step S341: scoring the k retrospective images, the scoring formula is:

[0085]

[0086] Among them, n represents the number of electric bicycles, m represents the number of combustibles, t represents the time of backtracking the current distance, F is the sensitivity factor, which is used to adjust the influence of the number of electric bicycles and the number of combustibles, and G is the gravity factor, which is used to adjust the influence of time;

[0087] Step S342: Select the retrospective image with the highest score as the best retrospective image.

[0088] Specifically, by adjusting the sensitivity factor F and the gravity factor G, the weight of the scoring formula can be flexibly adjusted according to the actual situation to improve the accuracy of the scoring. At the same time, the scoring formula contains the time factor t, which can dynamically adjust the score to ensure that the selected image is timely and reflects the latest fire situation. Under other conditions, as time passes, the score of the retrospective image will continue to decrease. By comprehensively considering the number of electric bicycles, the number of combustibles and time factors, the misjudgment caused by a single factor can be reduced, and the best retrospective image can be determined through scoring to improve the reliability and stability of the system.

[0089] Furthermore, in step S400, the step of obtaining the fire risk parameters in the optimal retrospective image and determining the fire risk level of the electric bicycle according to the fire risk parameters and preset fire-related parameters includes:

[0090] Step S410: obtaining preset fire-related parameters, wherein the preset fire-related parameters include the number of electric bicycles designed to be parked and a disaster consequence score;

[0091] Step S420: determining the number score of electric bicycles according to the number of electric bicycles in the best backtracking image;

[0092] Step S430: calculating the density of electric bicycles according to the number of electric bicycles in the optimal backtracking image and the designed parking number of electric bicycles, and determining the overload coefficient score according to the density of electric bicycles;

[0093] Step S440: Calculate the fire risk value of electric bicycles according to the number score of electric bicycles, the overload coefficient score and the disaster consequence score, and output the corresponding danger level according to the fire risk value of the electric bicycles.

[0094] Specifically, the present invention obtains emergency resource scheduling related data and sets model parameters, wherein the camera risk judgment strategy C is the disaster consequence score, which is selected manually; D is the design parking quantity of electric bicycles, that is, the design index of the number of electric bicycles parked in the monitoring area or the national / local electric bicycle parking standard, which is filled in manually; F is the sensitivity factor, which is used to adjust the influence of the number of electric bicycles and the number of combustibles; G is the gravity factor, which is used to adjust the influence of time. The disaster consequence score C is set according to the following table:

[0095]

[0096]

[0097] According to the number N of electric bicycles in the optimal backtracking image bike , refer to the following table to determine the number fraction N of electric bicycles:

[0098]

[0099] According to the number N of electric bicycles in the optimal backtracking image bike The electric bicycle density is calculated by the number of electric bicycles designed to be parked D. The calculation formula for the electric bicycle density is as follows: Electric bicycle density

[0100] According to the density of the electric bicycle, the overload coefficient fraction O is determined by referring to the following table:

[0101]

[0102]

[0103] Finally, the fire risk value of electric bicycles is calculated based on the number score of electric bicycles, the overload coefficient score and the disaster consequence score. The electric bicycle fire risk value NOC = N × O × C, where N is the number score of electric bicycles, O is the overload coefficient score, and C is the disaster consequence score.

[0104] According to the risk value NOC, the corresponding danger level is output as follows:

[0105]

[0106] Furthermore, the step S400 further includes the following steps:

[0107] Step S500: Regularly check the online update conditions of the model according to a preset statistical period, and perform online update or continue monitoring according to the check result.

[0108] Specifically, a statistical cycle can be preset (such as once a week or once a month, adjusted as needed). Reasonable setting of the statistical cycle can ensure that the model is updated in time and avoid waste of resources caused by frequent updates. In each statistical cycle, the model performance is regularly evaluated to timely discover the degradation of the model. Setting clear online update conditions can ensure that the model is updated when necessary and avoid unnecessary update operations. If the online update conditions are met, the model is updated online using new data. Online updates can be performed through incremental learning or retraining. Online updates can adjust model parameters in time to adapt them to the new data distribution and improve the prediction accuracy and robustness of the model. Among them, incremental learning refers to incremental training based on the existing model using new data to update model parameters; retraining refers to retraining the model using a combination of new and old data to generate new model parameters. If the online update conditions are not met, continue to use the current model for monitoring and wait for the next statistical cycle.

[0109] Furthermore, in step S500, the online update condition of the model is regularly checked according to a preset statistical period, and the step of performing online update or continuing monitoring according to the check result includes:

[0110] Step S510: Count the prediction performance of the model according to a preset statistical period, and store the statistical results in a statistical database;

[0111] Step S520: Check whether the model meets one of the following online update conditions: the model online time reaches the preset update cycle threshold, the model performance index is lower than the preset threshold, and the prediction accuracy value of the historical N statistical cycles including the current statistical cycle continues to decrease. If any of the conditions is met, execute step S530. If none of them are met, continue to monitor the model status and wait for the next periodic check;

[0112] Step S530: Acquire the latest image as training data and perform online training on the model to obtain an updated model.

[0113] Specifically, you can preset a statistical cycle (such as once a week or once a month, and adjust as needed), use the scheduled task scheduling tool to regularly trigger online update condition checks, including performance indicators such as model accuracy, recall rate, F1 score, etc., and store the statistical results in the statistical database for subsequent analysis and inspection. Check whether the model meets one of the following online update conditions: the model online time reaches the preset update cycle threshold, the model's performance indicator is lower than the preset threshold, and the prediction accuracy value of the historical N statistical cycles including the current statistical cycle continues to decline. Among them, the model online time reaches the preset update cycle threshold check: record the current time when the model is online, and update the model's running time at each check, set a fixed update cycle threshold (for example, at least once every three months), calculate the current running time of the model at each check, and compare it with the preset update cycle threshold. If the model's running time reaches the preset update cycle threshold, the online update mechanism is triggered; the model's performance indicator is lower than the preset threshold check: in each statistical period, calculate the model's performance indicator and compare it with the preset threshold. If the performance indicator is lower than the preset threshold, the online update mechanism is triggered. Common model performance indicators include response time, memory usage, CPU usage, accuracy, precision, recall rate and F1 score. If the performance indicator is lower than the preset threshold, the online update mechanism is triggered; the prediction accuracy value of the historical N statistical periods including the current statistical period continues to decline check: obtain the prediction accuracy value of the historical N statistical periods including the current statistical period from the statistical database. Assuming N=3, obtain the prediction accuracy values ​​of the current statistical period (3rd period P3), the previous statistical period (2nd period P2) and the previous statistical period (1st period P1). If P1>=P2>=P3 holds, it is considered that the prediction accuracy value has continued to decline in the historical N statistical periods. When the model meets the online update conditions, the latest image set is obtained as training data to train the model online to improve the performance of the model. After the training is completed, the current model is replaced with the updated model to ensure that the performance of the model remains in the best state. It can be understood that by regularly evaluating the prediction performance of the model at a preset statistical period, it can ensure that the performance of the model is always under monitoring; by checking multiple conditions such as the model's online time, performance indicators and prediction accuracy values, it can ensure that the model can be updated in time when it needs to be updated; and regularly using the latest image data to train the model online can ensure that the model can adapt to the latest environmental changes and improve the prediction accuracy of the model.

[0114] Furthermore, in step S320, each item in the loss function of the electric bicycle identification and analysis model is specifically defined as follows:

[0115]

[0116] Among them, SmoothL1 is a smooth L1 loss function used to calculate the regression loss between the predicted box and the true box; P is the predicted box; B P It is the bounding box adjusted by regression based on the predicted box P; It is the real box with the maximum IoU intersection and union ratio with the predicted box P, that is is the size of the positive sample prediction box set, that is, the number of prediction boxes in the set;

[0117]

[0118] Among them, Smooth ln It is a smooth logarithmic loss function used to calculate the distance between the predicted box and the surrounding real box; is the other real box with the maximum IoU with the predicted box P, that is and For the intersection-over-union ratio, calculate the predicted box B P With real frame The overlapping area and the ground-truth box The ratio of the total area,

[0119]

[0120] Among them, L RepBox To exclude the predicted box items, the smooth logarithmic loss function Smooth is used ln ;∈ is a small constant used to prevent the denominator from being zero; P i and P j for Different prediction boxes in the positive sample prediction box set, and is the predicted box P i and P j The bounding box of is the indicator function, when Returns 1 if true, otherwise returns 0.

[0121]

[0122] Among them, L num is the quantity loss term, and num is the absolute value of the difference between the predicted number of electric bicycles in the image and the actual number of electric bicycles.

[0123] The above is a possible implementation of the loss function. It is understandable that the composition of each item of the loss function can also have other alternative forms. The following example illustrates:

[0124] Alternative formula (focal loss): L Attr=-α t (1-p t ) γ log(p t )

[0125] Alternative formula (squared loss):

[0126] Alternative formula (cosine similarity loss):

[0127] Alternative formula (binary cross entropy loss):

[0128] Through the above multiple implementations, the present invention is not limited to a specific loss function form, but covers a variety of possible loss function configurations. This diverse implementation not only improves the flexibility and adaptability of the model, but also expands the protection scope of the present invention, ensuring the effectiveness and robustness in different application scenarios.

[0129] Furthermore, in order to improve the robustness and generalization ability of the model, this embodiment provides a variety of data enhancement methods, which are applicable to the open flame and smoke recognition model, the fire load recognition model and the electric bicycle recognition analysis model. The specific data enhancement methods are as follows:

[0130] 1. Generate a random pixel mask within the annotation box (non-border area)

[0131] refer to Figure 8 In order to enhance the robustness of the open flame and smoke recognition model, the fire load recognition model, and the electric bicycle recognition and analysis model, we used the method of generating random pixel masks inside the annotation box for data augmentation. Specifically, for each original data, occluded pixels are randomly generated within the annotation box, and the occlusion ratio is randomly set between 30% and 80%, and the border area is not covered. Five enhanced images are generated for each original image. If the original dataset contains k images, the final generated dataset will contain 6k images.

[0132] 2. Cutting and assembling multiple flame image annotation frames

[0133] refer to Fig. 9 In order to further enrich the training dataset, we also used a method of cutting and assembling multiple flame image annotation boxes to generate additional augmented data. This method randomly selects 4 images from the augmented dataset, randomly selects a annotation box from each image for cropping, and then stretches and fills these 4 annotation boxes to combine them into a new 640x640 standard image. Through this method, an additional 2k augmented data can be generated from 6k augmented data, effectively improving the diversity of the data and the generalization ability of the model.

[0134] refer to Figure 7 , Fig.10 The second aspect of the present invention provides an electric bicycle fire degree judgment system based on image recognition, comprising:

[0135] A collection module is used to obtain the monitoring video stream of the electric bicycle parking point;

[0136] an identification module, for capturing a real-time image from the monitoring video stream, adding the real-time image to a retrospective image set according to a preset time interval, identifying the real-time image using a flame and smoke identification model, and determining whether a fire occurrence condition is met; if the fire occurrence condition is met, the control determination module determines the best retrospective image from the retrospective image set; if the fire occurrence condition is not met, the monitoring is continued;

[0137] A determination module, used to determine the best retrospective image from the retrospective image set;

[0138] A judgment module, used to obtain the fire risk parameters in the optimal retrospective image, and determine the fire hazard level of the electric bicycle according to the fire risk parameters and preset fire-related parameters;

[0139] The update module is used to regularly check the online update conditions of the model according to the preset statistical period, and perform online updates or continue monitoring based on the inspection results.

[0140] refer to Fig.11 The third aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for determining the degree of fire of an electric bicycle based on image recognition as described in the first aspect when executing the computer program.

[0141] The fourth aspect of the present invention also provides a computer-readable storage medium, on which is stored an electric bicycle fire severity judgment processing program based on image recognition. When the electric bicycle fire severity judgment processing program based on image recognition is executed by a processor, the steps of the electric bicycle fire severity judgment method based on image recognition as described in any of the above embodiments are implemented.

[0142] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electric bicycle fire degree judgment system based on image recognition, and uses various interfaces and lines to connect the various parts of the entire electric bicycle fire degree judgment system based on image recognition to process the executable device.

[0143] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electric bicycle fire degree judgment system based on image recognition by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0144] Compared with the prior art, the beneficial effects of the present invention include at least:

[0145] The present invention provides a method and system for judging the extent of fire of electric bicycles based on image recognition, wherein the method for judging the extent of fire of electric bicycles based on image recognition can more accurately identify the conditions for fire occurrence through the combined use of a flame smoke recognition model, an electric bicycle recognition analysis model and a combustible material recognition model, the flame smoke recognition model can effectively detect the smoke or flame generated by the initial fire, and the electric bicycle recognition analysis model can accurately locate the position and number of electric bicycles that may cause the fire; when the system detects a possible fire signal, it will select the best retrospective image that best reflects the fire condition from a series of images taken recently, and this process is achieved by calculating the image score, and the scoring criteria take into account factors such as the number, density and number of combustible materials of electric bicycles, so as to provide key information to decision makers in the shortest time and speed up the preparation and implementation of rescue. The present invention not only issues an alarm after discovering a fire, but more importantly, it can calculate a more accurate fire risk value based on information such as the number, density and number of combustible materials of electric bicycles, combined with preset fire-related parameters, and this risk value can be used to guide the rescue team to formulate a more reasonable response strategy. Furthermore, the system has the ability to learn and self-optimize online. By regularly checking the model's predictive performance and updating it online as needed, it ensures that the model is always in the best condition, can cope with the ever-changing environment and new challenges, and maintain a high level of recognition accuracy.

[0146] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and the equivalent structural transformations made by the contents of the present invention specification and drawings, or direct / indirect application in other related technical fields are all included in the scope of patent protection of the present invention. Therefore, the scope of protection of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for judging the degree of fire of an electric bicycle based on image recognition, characterized in that: The following steps are involved: Step S100: Acquire a surveillance video stream of an electric bicycle parking spot; Step S200: intercepting a real-time image from the monitoring video stream, adding the real-time image to the retrospective image set according to a preset time interval, using a flame and smoke recognition model to recognize the real-time image, and judging whether the fire occurrence condition is met. If the fire occurrence condition is met, executing step S300; if the fire occurrence condition is not met, continuing monitoring; Step S300: determining the best retrospective image from the retrospective image set; Step S400: Obtain fire risk parameters in the optimal retrospective image, and determine the fire hazard level of the electric bicycle according to the fire risk parameters and preset fire-related parameters.

2. The method for judging the degree of fire of an electric bicycle based on image recognition as claimed in claim 1, characterized in that: In step S300, the step of determining the best retrospective image from the retrospective image set includes: Step S310: obtaining the most recent k retrospective images from the retrospective image set, where k is a preset positive integer; Step S320: using the electric bicycle recognition analysis model to recognize the k retrospective images, and calculating the number of electric bicycles in each image; Step S330: using the combustible material recognition model to recognize the k retrospective images, and calculating the number of combustible materials in each image; Step S340: determining the best retrospective image according to the number, density and amount of combustibles of the k retrospective images.

3. The method for judging the degree of fire of an electric bicycle based on image recognition as claimed in claim 2, characterized in that: In step S340, determining the best retrospective image according to the number, density and quantity of combustibles of the k retrospective images comprises the following steps: Step S341: The k traceback images are scored, and the scoring formula is: Among them, n represents the number of electric bicycles, m represents the number of combustibles, t represents the time of backtracking the current distance, F is the sensitivity factor, which is used to adjust the influence of the number of electric bicycles and the number of combustibles, and G is the gravity factor, which is used to adjust the influence of time; Step S342: Select the retrospective image with the highest score as the best retrospective image.

4. The method for judging the degree of fire of an electric bicycle based on image recognition as claimed in claim 3, characterized in that: In step S400, the step of obtaining the fire risk parameters in the optimal retrospective image and determining the fire risk level of the electric bicycle according to the fire risk parameters and preset fire-related parameters includes: Step S410: obtaining preset fire-related parameters, wherein the preset fire-related parameters include the number of electric bicycles designed to be parked and a disaster consequence score; Step S420: determining the number score of electric bicycles according to the number of electric bicycles in the best backtracking image; Step S430: calculating the density of electric bicycles according to the number of electric bicycles in the optimal backtracking image and the designed parking number of electric bicycles, and determining the overload coefficient score according to the density of electric bicycles; Step S440: Calculate the fire risk value of electric bicycles according to the number score of electric bicycles, the overload coefficient score and the disaster consequence score, and output the corresponding danger level according to the fire risk value of the electric bicycles.

5. The method for judging the degree of fire of an electric bicycle based on image recognition according to any one of claims 1 to 4, characterized in that: The step S400 further includes the following steps: Step S500: Regularly check the online update conditions of the model according to a preset statistical period, and perform online update or continue monitoring according to the check result.

6. The method for judging the degree of fire of an electric bicycle based on image recognition as claimed in claim 5, characterized in that: In step S500, the online update condition of the model is regularly checked according to a preset statistical period, and the steps of performing online update or continuing monitoring according to the check result include: Step S510: Count the prediction performance of the model according to a preset statistical period, and store the statistical results in a statistical database; Step S520: Check whether the model meets one of the following online update conditions: the model online time reaches the preset update cycle threshold, the model performance index is lower than the preset threshold, and the prediction accuracy value of the historical N statistical cycles including the current statistical cycle continues to decrease. If any of the conditions is met, execute step S530. If none of them are met, continue to monitor the model status and wait for the next periodic check; Step S530: Acquire the latest image as training data and perform online training on the model to obtain an updated model.

7. The method for judging the degree of fire of an electric bicycle based on image recognition according to any one of claims 2 to 4, characterized in that: In step S320, the electric bicycle recognition analysis model is trained by an improved loss function, specifically: L=L Attr +a*L RepGT +β*L RepBox +γ*L num Among them, L is the total loss function, which is used to comprehensively evaluate the prediction performance of the model; L Attr is the attraction term, which is used to make the predicted box closer to the real box; L RepGT To exclude the real box item, it is used to make the predicted box away from the surrounding real box; L RepBox To exclude the prediction box item, it is used to keep the prediction box away from other surrounding prediction boxes; L num is the quantity loss term, which is used to make the number of predicted boxes closer to the number of real boxes; parameters α, β and γ are used to balance L RepGT , L RepBox , L num The weight of .

8. The method for judging the degree of fire of an electric bicycle based on image recognition as claimed in claim 7, characterized in that: In step S320, the specific definitions of the items in the loss function of the electric bicycle identification and analysis model are as follows: Among them, Smooth L1 is a smooth L1 loss function used to calculate the regression loss between the predicted box and the true box; P is the predicted box; B P It is the bounding box adjusted by regression based on the predicted box P; It is the real box with the maximum IoU intersection and union ratio with the predicted box P, that is is the size of the positive sample prediction box set, that is, the number of prediction boxes in the set; Among them, Smooth ln It is a smooth logarithmic loss function used to calculate the distance between the predicted box and the surrounding real box; is the other real box with the maximum IoU with the predicted box P, that is and For the intersection-over-union ratio, calculate the predicted box B P With real box The overlapping area and the ground-truth box The ratio of the total area, Among them, L RepBox To exclude the predicted box items, the smooth logarithmic loss function Smooth is used ln ;∈ is a small constant used to prevent the denominator from being zero; P i and P j for Different prediction boxes in the positive sample prediction box set, and is the predicted box P i and P j The bounding box of is the indicator function, when Returns 1 if true, otherwise returns 0. Among them, L num is the quantity loss term, and num is the absolute value of the difference between the predicted number of electric bicycles in the image and the actual number of electric bicycles.

9. An electric bicycle fire severity judgment system based on image recognition, characterized in that: include: A collection module is used to obtain the monitoring video stream of the electric bicycle parking point; an identification module, for capturing a real-time image from the monitoring video stream, adding the real-time image to a retrospective image set according to a preset time interval, identifying the real-time image using a flame and smoke identification model, and determining whether a fire occurrence condition is met; if the fire occurrence condition is met, the control determination module determines the best retrospective image from the retrospective image set; if the fire occurrence condition is not met, the monitoring is continued; A determination module, used to determine the best retrospective image from the retrospective image set; A judgment module, used to obtain the fire risk parameters in the optimal retrospective image, and determine the fire hazard level of the electric bicycle according to the fire risk parameters and preset fire-related parameters; The update module is used to regularly check the online update conditions of the model according to the preset statistical period, and perform online updates or continue monitoring based on the inspection results.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for determining the degree of fire of an electric bicycle based on image recognition as described in any one of claims 1 to 8 is implemented.

Images