Method and system for judging the degree of fire in electric bicycles based on image recognition

Abstract

The present invention discloses a method and system for determining the extent of an electric bicycle fire based on image recognition. The method includes the following steps: obtaining a surveillance video stream of an electric bicycle parking spot; intercepting real-time images from the surveillance video stream, adding the real-time images to a retrospective image set at preset time intervals, and using a flame and smoke recognition model to identify the real-time images to determine whether fire conditions are met. If the fire conditions are not met, monitoring is continued. If the fire conditions are met, the best retrospective image is determined from the retrospective image set; obtaining fire risk parameters from the best retrospective image, and determining the fire hazard level of the electric bicycle based on the fire risk parameters and preset fire-related parameters. The present invention can quantify the risk of electric bicycle fires, thereby improving the accuracy of fire hazard level determination.

Description

Technical Field

[0001] The present invention relates to the technical field of fire monitoring, and in particular to a method and system for judging the extent of fire in an electric bicycle based on image recognition. Background Art

[0002] With the rapid growth of the electric bicycle market in China, they have become an essential component of urban transportation, particularly popular among office workers, students, and housewives. However, the accompanying problem of electric bicycle fires has become increasingly prominent, threatening not only human safety but also causing significant economic losses. Existing electric bicycle fire prevention and control technologies primarily focus on the early warning stage, such as monitoring the status of charging devices and using smoke and flame detectors to identify incipient fires. While these measures can provide early warning of fires, they cannot automatically assess the fire's severity based on the specific circumstances. Therefore, rescuers must rely on personal experience and intuition to determine the severity of a fire. Furthermore, electric bicycle fires are characterized by dense smoke and bright flames, significantly impacting on-site surveillance footage after a fire breaks out. This hinders the performance of both manual identification and AI-based image recognition algorithms, directly impacting the effectiveness and timeliness of rescue strategies. Summary of the Invention

[0003] The main purpose of the present invention is to provide a method and system for judging the degree of fire of electric bicycles based on image recognition, aiming to solve the technical problem of improving the accuracy of judging the degree of fire hazard of electric bicycles in the background art.

[0004] To achieve the above-mentioned object, the present invention provides, in a first aspect, a method for determining the extent of fire in an electric bicycle based on image recognition, which is characterized by comprising the following steps:

[0005] Step S100: Acquire a surveillance video stream of an electric bicycle parking spot;

[0006] Step S200: capturing real-time images from the surveillance video stream, adding the real-time images to a retrospective image set at preset time intervals, and identifying the real-time images using a flame and smoke recognition model to determine whether fire occurrence conditions are met. If the fire occurrence conditions are met, executing step S300; if not, continuing monitoring;

[0007] Step S300: determining the best retrospective image from the retrospective image set;

[0008] Step S400: Obtain fire risk parameters in the optimal retrospective image, and determine the fire hazard level of the electric bicycle based on the fire risk parameters and preset fire-related parameters.

[0009] Furthermore, in step S300, the step of determining the best retrospective image from the retrospective image set includes:

[0010] Step S310: obtaining the most recent k retrospective images from the retrospective image set, where k is a preset positive integer;

[0011] Step S320: using the electric bicycle recognition analysis model to identify the k retrospective images and calculate the number of electric bicycles in each image;

[0012] Step S330: using the combustible material recognition model to identify the k retrospective images and calculate the number of combustible materials in each image;

[0013] Step S340: determining the best retrospective image according to the number, density and amount of combustible materials of the k retrospective images.

[0014] Furthermore, in step S340, determining the best retrospective image based on the number, density, and amount of combustible materials of the k retrospective images includes the following steps:

[0015] Step S341: Score the k retrospective images. The scoring formula is:

[0016]

[0017] Where n represents the number of electric bicycles, m represents the number of combustibles, t represents the time of backtracking the current distance, F is the sensitivity factor, which is used to adjust the influence of the number of electric bicycles and combustibles, and G is the gravity factor, which is used to adjust the influence of time.

[0018] Step S342: Select the retrospective image with the highest score as the best retrospective image.

[0019] Furthermore, in step S400, the step of obtaining the fire risk parameter in the optimal retrospective image and determining the fire hazard level of the electric bicycle according to the fire risk parameter and preset fire-related parameters includes:

[0020] Step S410: obtaining preset fire-related parameters, wherein the preset fire-related parameters include the designed parking quantity of electric bicycles and the disaster consequence score;

[0021] Step S420: determining an electric bicycle quantity score according to the number of electric bicycles in the optimal retrospective image;

[0022] Step S430: calculating the density of electric bicycles according to the number of electric bicycles in the optimal retrospective image and the designed parking number of electric bicycles, and determining the overload coefficient score according to the density of electric bicycles;

[0023] Step S440: Calculate the fire risk value of the electric bicycle according to the number score of the electric bicycles, the overload coefficient score and the disaster consequence score, and output the corresponding danger level according to the fire risk value of the electric bicycles.

[0024] Furthermore, the step S400 further includes the following steps:

[0025] Step S500: Regularly check the online update conditions of the model according to a preset statistical period, and perform online update or continue monitoring based on the check results.

[0026] Furthermore, in step S500, the steps of regularly checking the online update conditions of the model according to a preset statistical period and performing online update or continuing monitoring according to the check results include:

[0027] Step S510: Statistic the prediction performance of the model according to a preset statistical period, and store the statistical results in a statistical database;

[0028] Step S520: Check whether the model meets one of the following online update conditions: the model online time reaches the preset update cycle threshold, the model performance index is lower than the preset threshold, and the prediction accuracy value of the historical N statistical periods including the current statistical period continues to decline. If any of the conditions are met, execute step S530. If none of the conditions are met, continue to monitor the model status and wait for the next periodic check;

[0029] Step S530: Acquire the latest image as training data and perform online training on the model to obtain an updated model.

[0030] Furthermore, in step S320, the electric bicycle recognition analysis model is trained by an improved loss function, specifically:

[0031] L=L Attr +α*L RepGT +β*L RepBox +γ*L num

[0032] Among them, L is the total loss function, which is used to comprehensively evaluate the prediction performance of the model; L Attr is the attraction term, used to make the predicted box closer to the real box; L RepGT To exclude the real box item, it is used to keep the predicted box away from the surrounding real boxes; L RepBox To exclude the prediction box item, it is used to keep the prediction box away from other prediction boxes around it; L num is the quantity loss term, which is used to make the number of predicted boxes closer to the number of real boxes; the parameters α, β and γ are used to balance L RepGT 、L RepBox 、Lnum The weight of .

[0033] Furthermore, in step S320, each item in the loss function of the electric bicycle identification and analysis model is specifically defined as follows:

[0034]

[0035] Among them, Smooth L1 is a smooth L1 loss function used to calculate the regression loss between the predicted box and the true box; P is the predicted box; B P It is the bounding box adjusted by regression based on the predicted box P; It is the real box with the maximum IoU intersection and union ratio with the predicted box P, that is is the size of the positive sample prediction box set, that is, the number of prediction boxes in the set;

[0036]

[0037] Among them, Smooth ln It is a smooth logarithmic loss function used to calculate the distance between the predicted box and the surrounding real boxes; is the other real box with the largest IoU with the predicted box P, that is and For the intersection-over-union ratio, calculate the predicted box B P With real box The overlapping area and the ground truth box The ratio of the total area,

[0038]

[0039] Among them, L RepBox To exclude the predicted box items, the smooth logarithmic loss function Smooth is used ln ;∈ is a small constant used to prevent the denominator from being zero; P i and P j for Different prediction boxes in the positive sample prediction box set, and is the predicted box P i and P j The bounding box of is the indicator function, when Returns 1 if yes, otherwise returns 0.

[0040]

[0041] Among them, L numis the quantity loss term, and num is the absolute value of the difference between the predicted number of electric bicycles in the image and the actual number of electric bicycles.

[0042] A second aspect of the present invention provides an electric bicycle fire severity determination system based on image recognition, comprising:

[0043] An acquisition module is used to obtain the monitoring video stream of the electric bicycle parking point;

[0044] an identification module configured to capture real-time images from the surveillance video stream, add the real-time images to a retrospective image set at preset time intervals, identify the real-time images using a flame and smoke identification model, and determine whether fire occurrence conditions are met; if so, control the determination module to determine the best retrospective image from the retrospective image set; if not, continue monitoring;

[0045] A determination module, configured to determine an optimal retrospective image from the retrospective image set;

[0046] a judgment module, configured to obtain a fire risk parameter from the optimal retrospective image, and determine the fire hazard level of the electric bicycle based on the fire risk parameter and preset fire-related parameters;

[0047] The update module is used to regularly check the online update conditions of the model according to the preset statistical period, and perform online updates or continue monitoring based on the inspection results.

[0048] Furthermore, the electric bicycle fire severity judgment system based on image recognition also includes:

[0049] The update module is used to regularly check the online update conditions of the water pump group health status monitoring model according to a preset statistical period, and perform online update or continue monitoring according to the inspection results.

[0050] The third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that when the processor executes the computer program, the method for determining the degree of fire of an electric bicycle based on image recognition as described in the first aspect is implemented.

[0051] The fourth aspect of the present invention provides a computer-readable storage medium, on which is stored an electric bicycle fire degree judgment processing program based on image recognition. When the electric bicycle fire degree judgment processing program based on image recognition is executed by a processor, the steps of the electric bicycle fire degree judgment method based on image recognition as described in the first aspect are implemented.

[0052] Compared with the prior art, the beneficial effects of the present invention include at least:

[0053] The present invention provides a method and system for determining the extent of an electric bicycle fire based on image recognition. The method utilizes a combination of a flame and smoke recognition model, an electric bicycle identification and analysis model, and a combustible material recognition model to more accurately identify fire conditions. The flame and smoke recognition model effectively detects smoke or flames generated by an initial fire, while the electric bicycle identification and analysis model accurately locates the location and number of electric bicycles that may have caused the fire. When the system detects a possible fire signal, it selects the best retrospective image that best reflects the fire situation from a series of recently captured images. This process is achieved by calculating an image score that takes into account factors such as the number and density of electric bicycles and the amount of combustible materials. This provides decision makers with critical information in the shortest possible time, accelerating rescue preparation and implementation. The present invention not only issues an alarm upon fire detection but, more importantly, calculates a more accurate fire risk value based on information such as the number, density, and amount of combustible materials of electric bicycles, combined with preset fire-related parameters. This risk value can be used to guide rescue teams in developing more reasonable response strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived based on the structures shown in these drawings without inventive work. The purpose of the present invention, its functional features and advantages will be further explained in conjunction with the embodiments and with reference to the drawings.

[0055] Figure 1 A flowchart of a method for determining the extent of fire in an electric bicycle based on image recognition provided by an embodiment of the present invention;

[0056] Figure 2 A flow chart of a method for determining an optimal retrospective image provided by an embodiment of the present invention;

[0057] Figure 3 A scoring flow chart for determining the best retrospective image provided by an embodiment of the present invention;

[0058] Figure 4 A flow chart of a method for calculating the fire risk value of an electric bicycle provided in an embodiment of the present invention;

[0059] Figure 5 A flowchart of another method for determining the extent of fire in an electric bicycle based on image recognition provided by an embodiment of the present invention;

[0060] Figure 6 A flow chart of a method for periodically checking whether a model meets online update conditions according to a preset statistical period provided by an embodiment of the present invention;

[0061] Figure 7 A schematic diagram of the architecture of an electric bicycle fire severity determination model based on image recognition provided by an embodiment of the present invention;

[0062] Figure 8 A schematic diagram of generating a random pixel mask within a marked box provided by an embodiment of the present invention;

[0063] Figure 9 A schematic diagram of cutting and assembling multiple flame image annotation frames provided by an embodiment of the present invention;

[0064] Figure 10 A schematic diagram of an electric bicycle fire severity determination system based on image recognition provided by an embodiment of the present invention;

[0065] Figure 11 A schematic diagram of a computer device for judging the extent of an electric bicycle fire based on image recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0067] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0068] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the meaning of "and / or" appearing throughout the text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme in which A and B are satisfied at the same time. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0069] In view of this, the main purpose of the present invention is to provide a method and system for judging the degree of fire of electric bicycles based on image recognition, aiming to solve the technical problem of improving the accuracy of judging the degree of fire hazard of electric bicycles in the background technology.

[0070] To achieve the above-mentioned object, the present invention provides, in a first aspect, a method for determining the extent of fire in an electric bicycle based on image recognition, which is characterized by comprising the following steps:

[0071] Step S100: Acquire a surveillance video stream of an electric bicycle parking spot;

[0072] Step S200: capturing real-time images from the surveillance video stream, adding the real-time images to a retrospective image set at preset time intervals, and identifying the real-time images using a flame and smoke recognition model to determine whether fire occurrence conditions are met. If the fire occurrence conditions are met, executing step S300; if not, continuing monitoring;

[0073] Step S300: determining the best retrospective image from the retrospective image set;

[0074] Step S400: Obtain fire risk parameters in the optimal retrospective image, and determine the fire hazard level of the electric bicycle based on the fire risk parameters and preset fire-related parameters.

[0075] Specifically, refer to Figures 1-6The present invention obtains the monitoring video stream in real time through the high-definition camera installed at the electric bicycle parking point, and transmits the video stream to the central server for storage. A real-time image is intercepted from the monitoring video stream every second (which can be set according to the performance of the model and the equipment), and a real-time image is added to the retrospective image set according to a preset time interval (for example, every 20 minutes). Each real-time image is identified using a pre-trained flame and smoke recognition model to determine whether there is flame or smoke. If no flame or smoke is detected and the fire occurrence conditions are not met, monitoring continues; if flame or smoke is detected and the fire occurrence conditions are met, the best retrospective image is determined from the retrospective image set, and by analyzing each image in the retrospective image set, the image that best reflects the fire risk is selected as the best retrospective image. Fire risk parameters are extracted from the best retrospective image, including the number, density and amount of combustible materials of electric bicycles, etc. The electric bicycle fire risk value NOC is calculated based on the extracted fire risk parameters and preset fire-related parameters (such as the number of electric bicycles designed to be parked and the disaster consequence scoring standard), and the corresponding degree of danger is output according to the risk value NOC.

[0076] As you can see, by capturing real-time images from video and identifying flames and smoke, the system can promptly identify potential fire risks, improving the timeliness and effectiveness of fire warnings. By comprehensively considering factors such as the number and density of e-bikes and the amount of combustible materials, the system can comprehensively assess fire risks, provide more scientific judgments on the degree of fire danger, and facilitate the implementation of timely and effective response measures. An image is saved to the retrospective image collection every 20 minutes, ensuring that critical historical images are retained even during long-term monitoring processes, facilitating subsequent analysis and tracing.

[0077] Furthermore, in step S300, the step of determining the best retrospective image from the retrospective image set includes:

[0078] Step S310: obtaining the most recent k retrospective images from the retrospective image set, where k is a preset positive integer;

[0079] Step S320: using the electric bicycle recognition analysis model to identify the k retrospective images and calculate the number of electric bicycles in each image;

[0080] Step S330: using the combustible material recognition model to identify the k retrospective images and calculate the number of combustible materials in each image;

[0081] Step S340: determining the best retrospective image according to the number, density and amount of combustible materials of the k retrospective images.

[0082] Specifically, the most recent k retrospective images are obtained from the retrospective image set, where k is a preset positive integer (e.g., k=15). image=image1,image2,......,image k , image1 is the most recent image. Use the electric bicycle recognition analysis model to identify the k retrospective images and calculate the number of electric bicycles in each image, recorded as N bike =n1,n2,......,n k , n k image k The combustibles recognition model is used to identify the k retrospective images and the number of combustibles in each image is calculated, which is recorded as Combustibles = m1, m2, ..., m k , m k image k The number of combustible materials in the image, including clothing, cardboard boxes, and awnings, is calculated. The optimal retrospective image is then determined based on the number of e-bikes and combustible materials in the k retrospective images. Using the e-bike recognition and analysis model to calculate the number of e-bikes in each image allows for a more comprehensive assessment of fire risk; a greater number of e-bikes indicates a higher fire risk. Using the combustible material recognition model to calculate the number of combustible materials in each image allows for a more accurate assessment of the potential hazard of a fire; a greater number of combustible materials indicates a greater likelihood of fire spread. Selecting the optimal retrospective image from the most recent k images in the retrospective image set ensures that the selected image is up-to-date and reflects the latest fire situation.

[0083] Furthermore, in step S340, determining the best retrospective image based on the number, density, and amount of combustible materials of the k retrospective images includes the following steps:

[0084] Step S341: Score the k retrospective images. The scoring formula is:

[0085]

[0086] Where n represents the number of electric bicycles, m represents the number of combustibles, t represents the time of backtracking the current distance, F is the sensitivity factor, which is used to adjust the influence of the number of electric bicycles and combustibles, and G is the gravity factor, which is used to adjust the influence of time.

[0087] Step S342: Select the retrospective image with the highest score as the best retrospective image.

[0088] Specifically, by adjusting the sensitivity factor F and the gravity factor G, the scoring formula's weights can be flexibly adjusted based on actual conditions, improving scoring accuracy. Furthermore, the inclusion of a time factor, t, in the scoring formula allows for dynamic scoring adjustments, ensuring that the selected images are up-to-date and reflect the latest fire situation. With other conditions remaining unchanged, the scores of retrospective images decrease over time. By comprehensively considering the number of e-bikes, the amount of combustible materials, and time, misjudgments caused by a single factor are reduced, and the optimal retrospective image is determined through scoring, improving system reliability and stability.

[0089] Furthermore, in step S400, the step of obtaining the fire risk parameter in the optimal retrospective image and determining the fire hazard level of the electric bicycle according to the fire risk parameter and preset fire-related parameters includes:

[0090] Step S410: obtaining preset fire-related parameters, wherein the preset fire-related parameters include the designed parking quantity of electric bicycles and the disaster consequence score;

[0091] Step S420: determining an electric bicycle quantity score according to the number of electric bicycles in the optimal retrospective image;

[0092] Step S430: calculating the density of electric bicycles according to the number of electric bicycles in the optimal retrospective image and the designed parking number of electric bicycles, and determining the overload coefficient score according to the density of electric bicycles;

[0093] Step S440: Calculate the fire risk value of the electric bicycle according to the number score of the electric bicycles, the overload coefficient score and the disaster consequence score, and output the corresponding danger level according to the fire risk value of the electric bicycles.

[0094] Specifically, the present invention obtains emergency resource scheduling related data and sets model parameters, wherein the camera risk judgment strategy C is the disaster consequence score, which is manually selected; D is the designed number of electric bicycle parking spaces, which is the design indicator for the number of electric bicycles parked in the monitoring area or the national / local electric bicycle parking standard, which is manually filled in; F is the sensitivity factor, which is used to adjust the impact of the number of electric bicycles and the amount of combustibles; G is the gravity factor, which is used to adjust the impact of time. The disaster consequence score C is set according to the following table:

[0095]

[0096]

[0097] According to the number N of electric bicycles in the optimal backtracking image bike , refer to the following table to determine the number of electric bicycles fraction N:

[0098]

[0099] According to the number N of electric bicycles in the optimal backtracking image bike The electric bicycle density is calculated by the number of electric bicycles designed to be parked D. The calculation formula for the electric bicycle density is as follows: Electric bicycle density

[0100] According to the density of the electric bicycle, the overload coefficient fraction O is determined by referring to the following table:

[0101]

[0102]

[0103] Finally, the electric bicycle fire risk value is calculated based on the electric bicycle quantity score, overload coefficient score and disaster consequence score. The electric bicycle fire risk value NOC = N × O × C, where N is the electric bicycle quantity score, O is the overload coefficient score, and C is the disaster consequence score.

[0104] According to the risk value NOC, the corresponding danger level is output as follows:

[0105]

[0106] Furthermore, the step S400 further includes the following steps:

[0107] Step S500: Regularly check the online update conditions of the model according to a preset statistical period, and perform online update or continue monitoring based on the check results.

[0108] Specifically, a statistical cycle can be preset (such as weekly or monthly, adjusted as needed). Properly setting the statistical cycle ensures timely model updates while avoiding resource waste caused by frequent updates. During each statistical cycle, model performance should be regularly evaluated to promptly detect model degradation. Setting clear online update conditions ensures that the model is updated when necessary, avoiding unnecessary updates. If the online update conditions are met, the model is updated online using new data. Online updates can be performed through incremental learning or retraining. Online updates can timely adjust model parameters to adapt to the new data distribution, improving the model's predictive accuracy and robustness. Incremental learning refers to incremental training based on the existing model using new data to update model parameters; retraining refers to retraining the model using a combination of new and existing data to generate new model parameters. If the online update conditions are not met, monitoring continues using the current model until the next statistical cycle arrives.

[0109] Furthermore, in step S500, the steps of regularly checking the online update conditions of the model according to a preset statistical period and performing online update or continuing monitoring according to the check results include:

[0110] Step S510: Statistic the prediction performance of the model according to a preset statistical period, and store the statistical results in a statistical database;

[0111] Step S520: Check whether the model meets one of the following online update conditions: the model online time reaches the preset update cycle threshold, the model performance index is lower than the preset threshold, and the prediction accuracy value of the historical N statistical periods including the current statistical period continues to decline. If any of the conditions are met, execute step S530. If none of the conditions are met, continue to monitor the model status and wait for the next periodic check;

[0112] Step S530: Acquire the latest image as training data and perform online training on the model to obtain an updated model.

[0113] Specifically, you can preset a statistical cycle (such as weekly or monthly, and adjust as needed) and use a scheduled task scheduling tool to regularly trigger online update condition checks, including performance indicators such as the model's accuracy, recall rate, and F1 score. The statistical results are stored in a statistical database for subsequent analysis and inspection. Check whether the model meets one of the following online update conditions: the model's online time reaches the preset update cycle threshold, the model's performance indicator is lower than the preset threshold, and the prediction accuracy value of the historical N statistical cycles, including the current statistical cycle, continues to decline. Among them, the model online time reaches the preset update cycle threshold check: record the current time when the model is online, and update the model's running time at each check, set a fixed update cycle threshold (for example, update at least once every three months), and calculate the current running time of the model at each check, and compare it with the preset update cycle threshold. If the model's running time reaches the preset update cycle threshold, the online update mechanism is triggered; the model's performance indicator is lower than the preset threshold check: in each statistical period, calculate the model's performance indicator and compare it with the preset threshold. If the performance indicator is lower than the preset threshold, the online update mechanism is triggered. Common model performance indicators include response time, memory usage, CPU usage, accuracy, precision, recall rate and F1 score. If the performance indicator is lower than the preset threshold, the online update mechanism is triggered; the prediction accuracy value of the historical N statistical periods including the current statistical period continues to decline check: obtain the prediction accuracy value of the historical N statistical periods including the current statistical period from the statistical database. Assuming N=3, obtain the prediction accuracy values ​​of the current statistical period (3rd period P3), the previous statistical period (2nd period P2) and the previous statistical period (1st period P1). If P1>=P2>=P3 holds, it is considered that the prediction accuracy value has continued to decline in the historical N statistical periods. When the model meets the online update conditions, the latest image set is obtained as training data to train the model online to improve the performance of the model. After the training is completed, the current model is replaced with the updated model to ensure that the performance of the model remains in the best state. It can be understood that by regularly evaluating the prediction performance of the model at preset statistical periods, it can be ensured that the performance of the model is always under monitoring; by checking multiple conditions such as the model's online time, performance indicators and prediction accuracy values, it can be ensured that the model can be updated in time when it needs to be updated; and regularly using the latest image data to train the model online can ensure that the model can adapt to the latest environmental changes and improve the model's prediction accuracy.

[0114] Furthermore, in step S320, each item in the loss function of the electric bicycle identification and analysis model is specifically defined as follows:

[0115]

[0116] Among them, SmoothL1 is a smooth L1 loss function used to calculate the regression loss between the predicted box and the true box; P is the predicted box; B P It is the bounding box adjusted by regression based on the predicted box P; It is the real box with the maximum IoU intersection and union ratio with the predicted box P, that is is the size of the positive sample prediction box set, that is, the number of prediction boxes in the set;

[0117]

[0118] Among them, Smooth ln It is a smooth logarithmic loss function used to calculate the distance between the predicted box and the surrounding real boxes; is the other real box with the largest IoU with the predicted box P, that is and For the intersection-over-union ratio, calculate the predicted box B P With real box The overlapping area and the ground truth box The ratio of the total area,

[0119]

[0120] Among them, L RepBox To exclude the predicted box items, the smooth logarithmic loss function Smooth is used ln ;∈ is a small constant used to prevent the denominator from being zero; P i and P j for Different prediction boxes in the positive sample prediction box set, and is the predicted box P i and P j The bounding box of is the indicator function, when Returns 1 if yes, otherwise returns 0.

[0121]

[0122] Among them, L num is the quantity loss term, and num is the absolute value of the difference between the predicted number of electric bicycles in the image and the actual number of electric bicycles.

[0123] The above is one possible implementation of the loss function. It is understandable that the composition of each item of the loss function can also have other alternative forms. The following examples illustrate:

[0124] Alternative formula (focal loss): L Attr=-α t (1-p t ) γ log(p t )

[0125] Alternative formula (squared loss):

[0126] Alternative formula (cosine similarity loss):

[0127] Alternative formula (binary cross entropy loss):

[0128] Through the various implementations described above, the present invention is not limited to a specific loss function form, but covers a variety of possible loss function configurations. This diverse implementation not only improves the flexibility and adaptability of the model, but also expands the scope of protection of the present invention, ensuring its effectiveness and robustness in different application scenarios.

[0129] Furthermore, to improve the robustness and generalization ability of the model, this embodiment provides multiple data enhancement methods, which are applicable to the open flame and smoke recognition model, the fire load recognition model, and the electric bicycle recognition and analysis model. The specific data enhancement methods are as follows:

[0130] 1. Generate a random pixel mask within the annotation box (non-border area)

[0131] refer to Figure 8 To enhance the robustness of the open flame and smoke detection model, the fire load detection model, and the electric bicycle identification and analysis model, we employed data augmentation by generating random pixel masks within the annotated box. Specifically, for each original data point, occluded pixels were randomly generated within the annotated box, with the occlusion ratio randomly set between 30% and 80%, and the border area was not covered. Five augmented images were generated for each original image. If the original dataset contained k images, the resulting dataset would contain 6k images.

[0132] 2. Cutting and assembling multiple flame image annotation frames

[0133] refer to Figure 9 To further enrich the training dataset, we also used a method to generate additional augmented data by cutting and assembling the annotated boxes from multiple flame images. This method randomly selects four images from the augmented dataset, randomly selects a box from each image, and then crops these four boxes by stretching and filling them to form a new 640x640 standard image. This method can generate an additional 2k augmented data from 6k augmented data, effectively improving data diversity and model generalization.

[0134] refer to Figure 7 、 Figure 10 The second aspect of the present invention provides an electric bicycle fire severity determination system based on image recognition, comprising:

[0135] An acquisition module is used to obtain the monitoring video stream of the electric bicycle parking point;

[0136] an identification module configured to capture real-time images from the surveillance video stream, add the real-time images to a retrospective image set at preset time intervals, identify the real-time images using a flame and smoke identification model, and determine whether fire occurrence conditions are met; if so, control the determination module to determine the best retrospective image from the retrospective image set; if not, continue monitoring;

[0137] A determination module, configured to determine an optimal retrospective image from the retrospective image set;

[0138] a judgment module, configured to obtain a fire risk parameter from the optimal retrospective image, and determine the fire hazard level of the electric bicycle based on the fire risk parameter and preset fire-related parameters;

[0139] The update module is used to regularly check the online update conditions of the model according to the preset statistical period, and perform online updates or continue monitoring based on the inspection results.

[0140] refer to Figure 11 The third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for determining the degree of fire of an electric bicycle based on image recognition as described in the first aspect when executing the computer program.

[0141] The fourth aspect of the present invention also provides a computer-readable storage medium, which stores an electric bicycle fire degree judgment processing program based on image recognition. When the electric bicycle fire degree judgment processing program based on image recognition is executed by a processor, it implements the steps of the electric bicycle fire degree judgment method based on image recognition as described in any of the above embodiments.

[0142] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor serves as the control center of the electric bicycle fire severity determination system based on image recognition, and utilizes various interfaces and lines to connect the various parts of the entire electric bicycle fire severity determination system processing and operating devices based on image recognition.

[0143] The memory can be used to store the computer programs and / or modules. The processor implements the various functions of the electric bicycle fire severity determination system based on image recognition by running or executing the computer programs and / or modules stored in the memory and accessing the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory can include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0144] Compared with the prior art, the beneficial effects of the present invention include at least:

[0145] The present invention provides a method and system for determining the extent of an electric bicycle fire based on image recognition. The method utilizes a combination of a flame and smoke recognition model, an electric bicycle identification and analysis model, and a combustible material recognition model to more accurately identify fire conditions. The flame and smoke recognition model effectively detects smoke or flames generated by an initial fire, while the electric bicycle identification and analysis model accurately locates the location and number of electric bicycles that may have caused the fire. When the system detects a possible fire signal, it selects the best retrospective image that best reflects the fire situation from a series of recently captured images. This process is achieved by calculating an image score that takes into account factors such as the number and density of electric bicycles and the amount of combustible materials. This provides decision makers with critical information in the shortest possible time, accelerating rescue preparation and implementation. The present invention not only issues an alarm upon fire detection but, more importantly, calculates a more accurate fire risk value based on information such as the number, density, and amount of combustible materials of electric bicycles, combined with preset fire-related parameters. This risk value can be used to guide rescue teams in developing more reasonable response strategies. Furthermore, the system has the ability to learn and self-optimize online. By regularly checking the model's predictive performance and updating it online as needed, it ensures that the model is always in the best condition, can cope with the ever-changing environment and new challenges, and maintain high recognition accuracy.

[0146] The above-described embodiments merely represent several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that, for those skilled in the art, without departing from the concept of the present invention, several variations and improvements can be made, and equivalent structural transformations made using the contents of the present invention's description and drawings, or direct / indirect application in other related technical fields are all included within the scope of the present invention's patent protection. Therefore, the scope of protection of the present invention's patent shall be based on the appended claims.

Claims

1. A method for judging the extent of fire in electric bicycles based on image recognition, characterized in that: The following steps are involved: Step S100: Acquire a surveillance video stream of an electric bicycle parking spot; Step S200: capturing real-time images from the surveillance video stream, adding the real-time images to a retrospective image set at preset time intervals, and identifying the real-time images using a flame and smoke recognition model to determine whether fire occurrence conditions are met. If the fire occurrence conditions are met, executing step S300; if not, continuing monitoring; Step S300: determining the best retrospective image from the retrospective image set; Step S400: obtaining fire risk parameters in the optimal retrospective image, and determining the fire hazard level of the electric bicycle according to the fire risk parameters and preset fire-related parameters; In step S300, the step of determining the best retrospective image from the retrospective image set includes: Step S310: obtaining the most recent k retrospective images from the retrospective image set, where k is a preset positive integer; Step S320: using the electric bicycle recognition analysis model to identify the k retrospective images and calculate the number of electric bicycles in each image; Step S330: using the combustible material recognition model to identify the k retrospective images and calculate the number of combustible materials in each image; Step S340: determining the best retrospective image based on the number, density and amount of combustible materials of the k retrospective images; In step S340, determining the optimal retrospective image based on the number, density, and amount of combustible materials of the k retrospective images includes the following steps: Step S341: Score the k retrospective images. The scoring formula is: Where n represents the number of electric bicycles, m represents the number of combustibles, t represents the time of backtracking the current distance, F is the sensitivity factor, which is used to adjust the influence of the number of electric bicycles and combustibles, and G is the gravity factor, which is used to adjust the influence of time. Step S342: Select the retrospective image with the highest score as the best retrospective image.

2. The method for determining the extent of fire in an electric bicycle based on image recognition according to claim 1, wherein: In step S400, the steps of obtaining the fire risk parameters in the optimal retrospective image and determining the fire hazard level of the electric bicycle according to the fire risk parameters and preset fire-related parameters include: Step S410: obtaining preset fire-related parameters, wherein the preset fire-related parameters include the designed parking quantity of electric bicycles and the disaster consequence score; Step S420: determining an electric bicycle quantity score according to the number of electric bicycles in the optimal retrospective image; Step S430: calculating the density of electric bicycles according to the number of electric bicycles in the optimal retrospective image and the designed parking number of electric bicycles, and determining the overload coefficient score according to the density of electric bicycles; Step S440: Calculate the fire risk value of the electric bicycle according to the number score of the electric bicycles, the overload coefficient score and the disaster consequence score, and output the corresponding danger level according to the fire risk value of the electric bicycles.

3. The method for determining the extent of fire in an electric bicycle based on image recognition according to any one of claims 1 to 2, characterized in that: The step S400 further includes the following steps: Step S500: Regularly check the online update conditions of the model according to a preset statistical period, and perform online update or continue monitoring based on the check results.

4. The method for determining the extent of fire in an electric bicycle based on image recognition according to claim 3, wherein: In step S500, the steps of regularly checking the online update conditions of the model according to a preset statistical period and performing online update or continuing monitoring according to the check results include: Step S510: Statistic the prediction performance of the model according to a preset statistical period, and store the statistical results in a statistical database; Step S520: Check whether the model meets one of the following online update conditions: the model online time reaches the preset update cycle threshold, the model performance index is lower than the preset threshold, and the prediction accuracy value of the historical N statistical periods including the current statistical period continues to decline. If any of the conditions are met, execute step S530. If none of the conditions are met, continue to monitor the model status and wait for the next periodic check; Step S530: Acquire the latest image as training data and perform online training on the model to obtain an updated model.

5. The method for determining the extent of fire in an electric bicycle based on image recognition according to any one of claims 1 to 2, characterized in that: In step S320, the electric bicycle recognition analysis model is trained by an improved loss function, specifically: L=L Attr +a*L RepGT +β*L RepBox +γ*L num Among them, L is the total loss function, which is used to comprehensively evaluate the prediction performance of the model; L Attr is the attraction term, used to make the predicted box closer to the real box; L RepGT To exclude the real box item, it is used to keep the predicted box away from the surrounding real boxes; L RepBox To exclude the prediction box item, it is used to keep the prediction box away from other prediction boxes around it; L num is the quantity loss term, which is used to make the number of predicted boxes closer to the number of real boxes; the parameters α, β and γ are used to balance L RepGT , L RepBox , L num The weight of .

6. The method for determining the extent of fire in an electric bicycle based on image recognition according to claim 5, characterized in that: In step S320, the specific definitions of each item in the loss function of the electric bicycle identification and analysis model are as follows: in, is a smooth L1 loss function used to calculate the regression loss between the predicted box and the true box; P is the predicted box; B P It is the bounding box adjusted by regression based on the predicted box P; It is the real box with the maximum IoU intersection and union ratio with the predicted box P, that is is the size of the positive sample prediction box set, that is, the number of prediction boxes in the set; Among them, Smooth ln It is a smooth logarithmic loss function used to calculate the distance between the predicted box and the surrounding real boxes; is the other real box with the largest IoU with the predicted box P, that is and For the intersection-over-union ratio, calculate the predicted box B P With real box The overlapping area and the ground truth box The ratio of the total area, Among them, L RepBox To exclude the predicted box items, the smooth logarithmic loss function Smooth is used ln ;∈ is a small constant used to prevent the denominator from being zero; P i and P j for Different prediction boxes in the positive sample prediction box set, and is the predicted box P i and P j The bounding box of is the indicator function, when Returns 1 if yes, otherwise returns 0; Among them, L num is the quantity loss term, and num is the absolute value of the difference between the predicted number of electric bicycles in the image and the actual number of electric bicycles.

7. An electric bicycle fire severity judgment system based on image recognition, characterized in that: include: An acquisition module is used to obtain the monitoring video stream of the electric bicycle parking point; an identification module configured to capture real-time images from the surveillance video stream, add the real-time images to a retrospective image set at preset time intervals, identify the real-time images using a flame and smoke recognition model, and determine whether a fire occurrence condition is satisfied. If so, step S300 is executed; if not, monitoring is continued; A determination module, configured to determine an optimal retrospective image from the retrospective image set; a judgment module, configured to obtain a fire risk parameter from the optimal retrospective image, and determine the fire hazard level of the electric bicycle based on the fire risk parameter and preset fire-related parameters; The update module is used to regularly check the online update conditions of the model according to the preset statistical period, and perform online updates or continue monitoring based on the inspection results; In the determination module, the step of determining the best retrospective image from the retrospective image set includes: Step S310: obtaining the most recent k retrospective images from the retrospective image set, where k is a preset positive integer; Step S320: using the electric bicycle recognition analysis model to identify the k retrospective images and calculate the number of electric bicycles in each image; Step S330: using the combustible material recognition model to identify the k retrospective images and calculate the number of combustible materials in each image; Step S340: determining the best retrospective image based on the number, density and amount of combustible materials of the k retrospective images; In step S340, determining the optimal retrospective image based on the number, density, and amount of combustible materials of the k retrospective images includes the following steps: Step S341: Score the k retrospective images. The scoring formula is: Where n represents the number of electric bicycles, m represents the number of combustibles, t represents the time of backtracking the current distance, F is the sensitivity factor, which is used to adjust the influence of the number of electric bicycles and combustibles, and G is the gravity factor, which is used to adjust the influence of time. Step S342: Select the retrospective image with the highest score as the best retrospective image.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for determining the extent of fire in an electric bicycle based on image recognition as described in any one of claims 1 to 6 is implemented.

Images