A vision-based gas station safety monitoring method

Abstract

The invention discloses a gas station safety monitoring method based on vision, and belongs to the technical field of video security monitoring; the technical route of the invention is, A, generating a neural network; B, using the neural network to monitor smoke, fire and other events from a compressed image during monitoring, and when a person is monitored, intercepting an image with a width w and a height h from a high-resolution original image, and then using the neural network to identify cigarettes, lighters, mobile phones and human limbs from the image, and then monitoring various events; when an event is monitored, using voice prompts and performing event processing in time; the beneficial effect of the invention is that the high-resolution original image can be used to implement accurate and reliable monitoring within the entire gas station, the safety hazards of the gas station can be discovered in advance, the safety of the gas station can be greatly improved, and the occurrence of major and serious malicious events and property losses can be greatly reduced; the invention can be widely used in gas station safety monitoring, and has good economic benefits and broad market prospects.

Description

Technical Field

[0001] The invention belongs to the technical field of video security monitoring, and in particular relates to a gas station safety monitoring method based on vision. Background Art

[0002] Chinese patent CN112699750A discloses a smart gas station safety monitoring method and system based on edge computing and AI. The monitoring method includes an edge computing unit in the gas station obtaining video images and heat source information in the gas station through a dual-spectrum camera; an image recognition module in the edge computing unit obtains video frames through video images, performs character segmentation on the obtained monitoring pictures, and obtains a character segmentation map; the edge computing unit analyzes the posture in the character segmentation map, and determines whether the character has dangerous actions in combination with the heat source information; if the character has dangerous actions, the edge computing unit immediately sends a warning to the site and the superior management center.

[0003] The shortcomings of the above system are: (1) When edge computing is used to detect events, the reliability of edge computing in detecting events will be greatly reduced when the external environment, especially the lighting environment, changes greatly; (2) When the camera is far away from the heat source, the size of the heat source in the image is small, and the thermal radiation is also small, so the probability and possibility of detecting the heat source are low; (3) The behavior of a person is judged only by the person's posture (such as raising a hand), and the monitoring method is single, and the probability and reliability of detecting events such as people smoking and making phone calls are low; (4) It does not specify the range and image resolution within which the monitoring system can reliably detect events such as people smoking and making phone calls; this is because if the monitoring range is large or the camera resolution is low, the person's posture cannot be effectively monitored; (5) It does not specify how to monitor people doing dangerous actions; (6) It does not monitor fire, smoke, cigarettes, lighters, mobile phones, and does not monitor events that are more dangerous, such as people with cigarettes in their mouths, people holding cigarettes, and people holding lighters. It also does not monitor events such as people holding mobile phones, people running, and people doing intense actions. Summary of the invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a gas station safety monitoring method based on vision; based on a neural network, a high-resolution image is used to accurately identify the target object, thereby effectively and reliably monitoring various events.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A gas station safety monitoring method based on vision, wherein the method is implemented by a gas station safety monitoring system, wherein the gas station safety monitoring system comprises N monitoring subsystems, a server, a GPU server, T network video recorders, a system switch, a network device and a system power supply, wherein the monitoring subsystem comprises an intelligent analyzer, a switch, a camera, a speaker and a power supply, wherein the steps of the method are as follows:

[0007] Step A, generating a neural network; the generating neural network includes acquiring a picture, using the picture to acquire a processing set, and using the processing set to train a neural network model to obtain a neural network capable of identifying a target object; wherein the picture is a picture containing a target object and the number of acquired pictures is ≥ PNum; PNum is a positive integer; acquiring a processing set using the picture includes labeling processing, correction processing, and compression processing, and obtaining a processing set consisting of a compressed picture and a labeling file of the compressed picture and a set target object picture and a labeling file of the set target object picture; the target object includes background targets and set targets; the background targets include at least a refueling island, a refueling machine, a column, a crash barrier, and a wall; the set targets include at least targets including people, cigarettes, lighters, mobile phones, and fire; the targets including people include at least people, people with cigarettes in their mouths, people holding cigarettes, people holding lighters, people holding mobile phones, people running, and people with abnormal limbs;

[0008] Step B: monitoring events;

[0009] The steps to monitor an event are as follows:

[0010] Step B.1, identifying the target object based on the neural network to obtain an identification set;

[0011] Step B.2, monitoring events according to the identification set; events refer to smoke, fire, and people with abnormal behavior; abnormal behavior includes holding a cigarette in the mouth, holding a cigarette, holding a lighter, holding a mobile phone, running, and abnormal limb state;

[0012] Step B.3: If an event is detected, go to step B.4; otherwise go to step B.1;

[0013] Step B.4: Event processing; after completing this step, go to step B.1.

[0014] The steps of obtaining the identification set by identifying the target object based on the neural network of the present invention are as follows:

[0015] The intelligent analyzer captures a frame of image Img from the video image output by the camera; wherein the width of the image Img is W and the height is H, and W and H need to satisfy formula (1),

[0016] W ≥ Dw × RegPnX / MinW

[0017] H≥Dh×RegPnY / MinH (1)

[0018] (1) Wherein, Dw and Dh are the width and height of the camera monitoring area, respectively; RegPnX and RegPnY are the minimum pixel points in the X direction and the minimum pixel points in the Y direction of the object recognized by the neural network, respectively; MinW and MinH are the width and height of the minimum object that the neural network needs to recognize from the image Img, and MinW and MinH correspond to the actual width in the X direction and the actual height in the Y direction monitored by the camera, respectively;

[0019] Compress the image Img into a compressed image; the compressed image is an image with a width of w and a height of h; w and h are the width and height of the image processed by the neural network, respectively, and w and h must satisfy formula (2),

[0020] w≥MinW×W / ThingW

[0021] h≥MinH×H / ThingH (2)

[0022] (2) In the formula, ThingW and ThingH are the width and height of the smallest object that the neural network needs to recognize from the compressed image, and ThingW and ThingH correspond to the actual width in the X direction and the actual height in the Y direction monitored by the camera, respectively;

[0023] Based on the neural network, the target objects are identified from the compressed image and the identification set is obtained; the elements in the identification set at least include the number of background target objects identified from the compressed image, the minimum circumscribed rectangle that frames each target object, and the identifier of each target object.

[0024] The steps of monitoring events according to the identification set are as follows:

[0025] According to the identification set, an event with smoke is monitored; the event with smoke is monitored when the number of background targets in the identification set is less than the maximum number of background targets; the maximum number of background targets is the number of background targets in the identification set when there is no smoke;

[0026] According to the identification set, monitoring the fire event; the fire event is monitored when at least one target object identifier in the identification set is an identifier of the target object being fire;

[0027] According to the identification set, events of people with abnormal behavior are monitored; events of people with abnormal behavior are monitored when at least one target object identifier in the identification symbol set is an identifier of a person with abnormal behavior, or the ASM condition must be met; the identification symbol set is a set of identifiers of target objects including people obtained from the identification set; the ASM condition means that if the abnormal behavior is running or abnormal limb state, the number of times the abnormal behavior is monitored within the set time SetTime must be ≥ NumK; wherein, 2s≤SetTime≤5s; NumK=SetTime×2times / s.

[0028] The event processing includes generating event records, uploading event records, and saving event records. The event processing also includes safety prompts and event alarms.

[0029] The steps of generating a neural network are as follows:

[0030] Step 5.1, set the number of acquired pictures PNum, and set the processing set to an empty set; wherein PNum is a positive integer;

[0031] Step 5.2, obtaining pictures; the pictures are pictures containing the target object and the number of obtained pictures is ≥ PNum;

[0032] Step 5.3, obtaining a processing set using the image;

[0033] Step 5.4, splitting the processing set into a training set and a test set;

[0034] Step 5.5, training the neural network model using the training set;

[0035] Step 5.6, using the test set to test the neural network model and counting the test data of each target object; the test data of the target object refers to the number of target object tests and the number of correct tests of the target object;

[0036] Step 5.7, calculating the test accuracy of each target object according to the test data of each target object;

[0037] Step 5.8: If the accuracy of each target test is ≥ the set accuracy, go to step 5.10; otherwise, go to step 5.9;

[0038] Step 5.9, increase the PNum value, and then go to step 5.2; increasing the PNum value means that the PNum after increase = the PNum before increase × AddScale; wherein, 1.2≤AddScale≤2.0;

[0039] Step 5.10, compiling the neural network model into a neural network; the neural network is a neural network that can recognize the target object;

[0040] Step 5.11, write the neural network into each intelligent analyzer in the system.

[0041] The steps of obtaining the picture are as follows:

[0042] Acquire pictures from video recordings; wherein the video recordings are video recordings recorded by the network video recorder; and the pictures are pictures containing the target object and are different from each other;

[0043] Get images containing the specified target object from the external network.

[0044] The steps of obtaining the processing set using the image are as follows:

[0045] Annotation processing: Annotation processing refers to automatically annotating the unannotated pictures in the pictures using automatic annotation software to obtain the annotation file of the pictures;

[0046] Correction processing: Correction processing refers to using modification annotation software to correct incorrect annotation files into correct annotation files;

[0047] Compression processing; compression processing refers to compressing an uncompressed image in the image to obtain a compressed image of the image, and obtaining a labeling file of the compressed image according to the labeling file of the image;

[0048] Using each set of pictures and annotation files, obtain a set target picture and an annotation file of the set target picture; the set of pictures and annotation files refers to a picture in the set and an annotation file of the picture; the picture in each set of pictures refers to a picture in the pictures; the set target picture is a picture that contains at least one set target and has a width of w and a height of h;

[0049] A processing set is composed of each group of compressed images and annotation files and each group of set target object images and annotation files.

[0050] The steps of obtaining the set target object image and the set target object image's annotation file using each set of images and annotation files are as follows:

[0051] Use each set of images and annotation files to obtain the set target object images;

[0052] Using the set target object image, obtain a labeling file of the set target object image;

[0053] The steps of obtaining the labeling file of the set target object image using the set target object image are as follows:

[0054] Automatically annotate each unannotated set target object image using automatic annotation software to obtain an annotation file of the set target object image;

[0055] Use the annotation modification software to correct the incorrectly set target object image annotation file into a correct annotation file.

[0056] Compared with the prior art, the technical solution of the present invention is to use a neural network to monitor events such as smoke and fire in a compressed image. When a person is monitored, an image containing the entire person with a width of w and a height of h is captured from the high-resolution original image, and then the neural network identifies cigarettes, lighters, mobile phones, and human limbs from the image, thereby monitoring various events, rather than only monitoring by recognizing the posture of the person; therefore, the technical solution of the present invention greatly improves the extensiveness and reliability of monitoring, and has good practicality, accuracy, and timeliness; this solution generates event records, uploads event records, saves event records, and promptly issues safety prompts or event alarms when an event is monitored, which is of great practical significance for timely handling of events and reducing the occurrence of major and serious malicious events and property losses; the beneficial effect of the present invention is that various events can be accurately and reliably monitored within the entire gas station using high-resolution images, and potential safety hazards of the gas station can be discovered in advance, greatly improving the safety of the gas station; the present invention can be widely used in safety monitoring of gas stations and places with high safety requirements, and has good economic benefits and a broad market prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the structure of a gas station safety monitoring system according to an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of the structure of a monitoring subsystem according to an embodiment of the present invention;

[0059] Figure 3 A flow chart of monitoring events according to an embodiment of the present invention;

[0060] Figure 4 A flowchart of monitoring events according to an identification set according to an embodiment of the present invention;

[0061] Figure 5 A flowchart of generating a neural network according to an embodiment of the present invention;

[0062] Figure 6 This is an example diagram of obtaining a diagram containing a person from an original diagram according to an embodiment of the present invention.

[0063] The names of the numbers in the figure are as follows: 1. A person with a cigarette in his mouth; 2. A person with abnormal limbs;

[0064] Figure 6In the figure, PI(1) is an image containing the entire person 1 with a cigarette in his mouth and having a width of w and a height of h, GRect(1) is the minimum circumscribed rectangle of the person 1 with a cigarette in his mouth, GRC(1), GRw(1), and GRh(1) are the center, width, and height of GRect(1), respectively; wherein GRw(1) < w and GRh(1) < h; ExpPI(2) is an image containing the entire person 2 with abnormal limbs, GRect(2) is the minimum circumscribed rectangle of the person 2 with abnormal limbs, GRC(2), GRw(2), and GRh(2) are the center, width, and height of GRect(2), respectively; the width and height of the image ExpPI(2) are ExpGRw(2) and ExpGRh(2), respectively; wherein GRh(2) > h. DETAILED DESCRIPTION

[0065] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to make the present invention more comprehensive and complete and to fully convey the concepts of example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and their repeated description will be omitted.

[0066] The words expressing positions and directions described in the present invention are all explained with reference to the accompanying drawings as examples, but they can be changed as needed, and all such changes are included in the protection scope of the present invention.

[0067] Before introducing the embodiment scheme, the parameters and terms used in the embodiment are first introduced:

[0068] Set: A set is a set composed of some elements. The number of elements in the set Set is represented by |Set|; b∈Set means that b belongs to Set, or that b is an element in the set Set, ∈ means belongs to; the empty set Set means that there are no elements in the set Set, and |Set|=0.

[0069] Image coordinate system: The image coordinate system is a coordinate system with the upper left corner of the image as the origin O of the image coordinate system, the upper edge of the image as the X-axis and the positive X-axis pointing from point O to the right, and the left edge of the image as the Y-axis and the positive Y-axis pointing from point O to the bottom.

[0070] Video image: Video image refers to an image that is output continuously at a certain rate (such as 25 frames / s).

[0071] Embodiments 1 to 7 are a vision-based gas station safety monitoring method, wherein the method is implemented by a gas station safety monitoring system, the gas station safety monitoring system includes N monitoring subsystems, a server, a GPU server, T network video recorders, a system switch, a network device and a system power supply, the monitoring subsystem includes an intelligent analyzer, a switch, a camera, a speaker and a power supply, and the steps of the method are as follows:

[0072] Step A, generating a neural network; the generating the neural network includes acquiring a picture, using the picture to acquire a processing set, and using the processing set to train a neural network model to obtain a neural network that can identify a target object; wherein the picture is a picture containing a target object and the number of acquired pictures is ≥ PNum; PNum is a positive integer; acquiring the processing set using the picture includes labeling processing, correction processing, and compression processing, and obtaining a processing set consisting of a compressed picture and a labeling file of the compressed picture and a set target picture and a labeling file of the set target picture; the target object includes a background target object and a set target object; the background target object includes at least a refueling island, a refueling machine, a column, a crash barrier, and a wall; the set ... crash barrier, and a wall. At least includes a target object containing a person, a cigarette, a lighter, a mobile phone, and a fire; the target object containing a person at least includes a person, a person with a cigarette in his mouth, a person holding a cigarette, a person holding a lighter, a person holding a mobile phone, a person running, and a person with abnormal limbs; the person is not a person with abnormal behavior, and people with abnormal behavior include a person with a cigarette in his mouth, a person holding a cigarette, a person holding a lighter, a person holding a mobile phone, a person running, and a person with abnormal limbs; a person running refers to a person running with both feet off the ground at the same time; an abnormal limb state refers to a person's limbs having at least one of the following states: the hand is clenched into a fist, the wrist is higher than the elbow, the ankle is straight, the ankle is higher than the knee, the angle between the forearm and the upper arm is less than 45°, and the angle between the thigh and the upper body is less than 135°;

[0073] Embodiment 1 and embodiment 2 generate a neural network;

[0074] In Example 2, the completion time of generating the neural network is 2024 / 04 / 28 / 17:50:03;

[0075] Step B: monitoring events;

[0076] Embodiment 3 to Embodiment 7, monitoring events;

[0077] In this embodiment, the start time is 2024 / 04 / 29 / 08:30:05; the monitoring process is as shown in the attached Figure 3 As shown;

[0078] The steps to monitor an event are as follows:

[0079] Step B.1, identifying the target object based on the neural network to obtain an identification set;

[0080] Embodiments 3 to 7, identifying the target object based on a neural network to obtain an identification set;

[0081] Step B.2, monitoring events according to the identification set; events refer to smoke, fire, and people with abnormal behavior; abnormal behavior includes holding a cigarette in the mouth, holding a cigarette, holding a lighter, holding a mobile phone, running, and abnormal limb state;

[0082] Embodiments 3 to 7 monitor events according to the identification set;

[0083] Step B.3: If an event is detected, go to step B.4; otherwise go to step B.1;

[0084] In Example 3 to Example 6, an event is detected and the process goes to step B.4;

[0085] Example 7: No event is detected, go to step B.1;

[0086] Step B.4: event processing; after completing this step, go to step B.1;

[0087] Embodiment 3 to Embodiment 6, event processing; after this step is completed, proceed to step B.1.

[0088] In this embodiment, the gas station safety monitoring system includes N monitoring subsystems, a server, a GPU server, T network video recorders, a system switch, a network device and a system power supply; wherein,

[0089] The server is connected to the system switch via a network cable, and is connected to the following devices via the system switch and the network cable: (1) connected to the GPU server; (2) connected to each of the T network video recorders; (3) connected to the network device, and connected to the external network via the network device; (4) connected to the switch in each of the N monitoring subsystems, and connected to the intelligent analyzer via the switch; the server is a server that meets the use requirements and has a hard disk, an input device, and a display. The hard disk of the server is installed with a Windows operating system, a database MySQL, and a first application software. The server is used to manage each monitoring subsystem in the system and process the information and data of each monitoring subsystem; the first application software includes at least the following software and programs: database management software, video download software, screenshot software, picture management software, picture filtering program, automatic annotation software, annotation modification software, compression program, splitting program, voice output program, and network communication program; in this embodiment, N=24, T=3;

[0090] The GPU server is a GPU server that meets the use requirements and has a hard disk on which a Linux operating system and a second application software are installed, and has an input device and a display, and is used for training neural network models, testing neural network models, compiling neural network models, deep learning, and video encoding and decoding. The second application software includes at least the following software and programs: a neural network model training program, a neural network model testing program, a compiling software, and a network communication program;

[0091] The camera in each of the N monitoring subsystems in the system monitors a refueling area, and the N refueling areas cover the entire area of ​​the entire gas station;

[0092] The network video recorders connected to M(t) cameras in the T network video recorders are connected to the cameras in the M(t) monitoring subsystems through the system switch and the switch, and the network video recorders record the video of each of the M(t) cameras connected thereto according to a set time period, where M(t) is a positive integer, t=1,2,...,T, and M(1)+M(2)+...+M(T)=N; in this embodiment, M(1)=M(2)=M(3)=8; the set time period is 24 hours a day;

[0093] The network device is one of the following devices or a combination of devices: a switch, a fiber optic transceiver, a switch and a fiber optic transceiver, a wireless network device, a switch and a wireless network device, and the wireless network device includes a wireless router, a wireless bridge, and a wireless wifi device; in this embodiment, the network device is a switch and a fiber optic transceiver;

[0094] The system power supply is connected to the system switch and network equipment and provides the required DC power for the system switch and network equipment;

[0095] The servers, GPU servers, T network video recorders, system switches, network equipment and system power supplies are all placed indoors; Figure 1 As shown;

[0096] The monitoring subsystem includes an intelligent analyzer, a switch, a camera, a speaker and a power supply; the intelligent analyzer is connected to the speaker and the switch, and is connected to the camera and the system switch through the switch, and is connected to the server through the switch and the system switch; the power supply is connected to the intelligent analyzer, the switch, the camera, the speaker, and provides the required DC power for the intelligent analyzer, the switch, the camera, and the speaker; wherein the power cords for providing power to the camera and the speaker are explosion-proof power cords; as shown in the attached Figure 2 As shown;

[0097] The intelligent analyzer is an intelligent analyzer that meets the use requirements and has a hard disk and a Linux operating system and a third application software installed on the hard disk. The third application software at least includes the following software and programs: download video software, image processing program, Yolov8 neural network program, picture shooting program, video recording program, voice output program, network communication program; the image processing program at least includes a program for intercepting images, a program for compressing images, a program for expanding images, a program for obtaining image areas, a program for obtaining the intersection of image areas, and a program for obtaining the minimum circumscribed rectangle of a target object; the Yolov8 neural network is a neural network that can at least identify each of the target objects;

[0098] In this embodiment, the camera in each of the N monitoring subsystems is a camera with ≥15 million pixels, each camera monitors a refueling area with a width of ≥8.38m and a height of ≥6.25m, and the area monitored by the N cameras covers the entire area of ​​the gas station;

[0099] The intelligent analyzer, switch and power supply are all placed indoors;

[0100] The explosion-proof grade of the camera and the speaker is ExdIIBT4.

[0101] In embodiments 3 to 7, the steps of identifying the target object based on the neural network and obtaining the identification set are as follows:

[0102] The intelligent analyzer captures a frame of image Img from the video image output by the camera; wherein the width of the image Img is W and the height is H, and W and H need to satisfy formula (1),

[0103] W ≥ Dw × RegPnX / MinW

[0104] H≥Dh×RegPnY / MinH (1)

[0105] (1) In the formula, Dw and Dh are the width and height of the camera monitoring area, respectively; RegPnX and RegPnY are the minimum pixel point in the X direction and the minimum pixel point in the Y direction for the neural network to recognize an object, respectively; MinW and MinH are the width and height of the minimum object that the neural network needs to recognize from the image Img, and MinW and MinH correspond to the actual width in the X direction and the actual height in the Y direction monitored by the camera, respectively;

[0106] In this embodiment, W and H are calculated using formula (1):

[0107] W≥Dw×RegPnX / MinW=8380mm×4 / 7.5mm=4469.33

[0108] H≥Dh×RegPnY / MinH=6250mm×4 / 7.5mm=3333.33 (1)

[0109] (1) Where Dw=8.38m, Dh=6.25m; RegPnX=4, RegPnY=4 (for example, using the P2 output layer of Yolov8 can identify 4×4 objects); MinW=7.5mm, MinH=7.5mm (because the minimum diameter of a cigarette is 7.5mm); In this embodiment, the camera uses a camera with ≥15 million pixels, and the width W of the captured image is ≥4472 and the height H is ≥3354. Under such an image resolution, the system using this method can accurately and reliably identify cigarettes, lighters, mobile phones, or objects with a width ≥7.5mm and a height ≥ 7.5mm objects; when N (e.g., 24) cameras cover the entire area of ​​the gas station, the system using this method can accurately and reliably detect cigarettes, lighters, mobile phones, and human limbs throughout the gas station, and can effectively detect events such as people with cigarettes in their mouths, people holding cigarettes, people holding lighters, people holding mobile phones, people running, and people with abnormal limbs;

[0110] Compress the image Img into a compressed image CompImg; the compressed image CompImg is an image with a width of w and a height of h; w and h are respectively the width and height of the image processed by the neural network, and w and h must satisfy formula (2),

[0111] w≥MinW×W / ThingW

[0112] h≥MinH×H / ThingH (2)

[0113] (2) Wherein, ThingW and ThingH are the width and height of the smallest object that the neural network needs to recognize from the compressed image CompImg, and ThingW and ThingH correspond to the actual width in the X direction and the actual height in the Y direction monitored by the camera, respectively;

[0114] In this embodiment, formula (2) is used to calculate w and h:

[0115] w≥MinW×W / ThingW=7.5mm×4472 / 56mm=598.93

[0116] h≥MinH×H / ThingH=7.5mm×3354 / 42mm=598.93 (2)

[0117] (2) In the formula, ThingW = 56 mm, ThingH = 42 mm, and the width w = 608 and height h = 608 of the image processed by the Yolov8 neural network. According to the calculation, the present method can identify the smallest object with a width of 56 mm and a height of 42 mm from the compressed image CompImg using the Yolov8 neural network. Therefore, the monitoring system using the present method can reliably detect fire and other objects with a width ≥ 56 mm and a height ≥ 42 mm within a range of 8.38 m and 6.25 m monitored by a camera. The size of such monitored objects is sufficient to meet the actual monitoring needs of gas stations for fire and other events.

[0118] Based on the neural network, the target object is identified from the compressed image, and the identification set is obtained; the elements in the identification set at least include the number of background targets identified from the compressed image, the minimum circumscribed rectangle that frames each target object, and the identifier of each target object; the identifier of the target object is the identifier of the target object identified by the neural network; the identifier of the target object k is a symbol assigned to the target object k that is different from other targets, k=1,2,...,K; wherein k is the target number, K is the number of targets, and K is a positive integer;

[0119] In this embodiment, based on the Yolov8 neural network, the target object is identified from the compressed image CompImg, and the identification set CompRegSet is obtained; CompRegSet={RegBGsN, CompGRect(i), CompRSym(i)|i=1,2,...,I}; the elements in the identification set CompRegSet include the number RegBGsN of background targets identified from the compressed image CompImg, the minimum bounding rectangle CompGRect(i) that frames each target object (i) and the identifier CompRSym(i) of each target object (i), i=1,2,...,I; i is the number of the target object formed in increasing order of the center point of the minimum bounding rectangle CompGRect(i); I is the number of targets in the identification set CompRegSet, and I is a positive integer; the identifier of the target object is the Y olov8 neural network recognizes the marker of the target object; the marker of target object k is a symbol Symbol(k)=k, k=1,2,...,K assigned to target object k that is different from other targets; the numbering formed in increasing order of the points refers to the numbering formed according to the y value of the points in the image coordinate system from small to large, and then according to the x value from small to large; in this embodiment, K=16; the targets 1, 2,..., 16 are respectively a refueling island, a refueling machine, a pillar, a crash barrier, a wall, a person, a person with a cigarette in his mouth, a person holding a cigarette, a person holding a lighter, a person holding a mobile phone, a running person, a person with abnormal limbs, a cigarette, a lighter, a mobile phone, and fire; the image coordinate system is a coordinate system with the upper left corner of the image as the origin O, the upper edge of the image as the X-axis and the positive X-axis pointing from the origin O to the left, the left edge of the image as the Y-axis and the positive Y-axis pointing from the origin O to the bottom; the image coordinate system refers to the image coordinate system of the compressed image CompImg.

[0120] In Example 3 to Example 7, the steps of monitoring events according to the identification set are as follows:

[0121] According to the identification set, an event with smoke is monitored; the event with smoke is monitored when the number of background objects in the identification set is less than the maximum number of background objects; the maximum number of background objects is the number of background objects in the identification set when there is no smoke;

[0122] Embodiment 3, monitoring smoke events according to the identification set;

[0123] According to the identification set, the steps for monitoring smoke events are as follows:

[0124] According to the recognition set CompRegSet, a maximum number MaxBGsN of background targets is obtained;

[0125] In this embodiment, at the time of 2024 / 04 / 29 / 08:30:05, this step is executed for the first time, and the maximum number of background targets MaxBGsN=RegBGsN=5 is set; wherein RegBGsN is the number of background targets in the recognition set CompRegSet when there is no smoke=5; this step is executed at a time after 2024 / 04 / 29 / 08:30:05, and MaxBGsN=

[0126] Max(MaxBGsN,RegBGsN); wherein RegBGsN is the number of background targets in the recognition set CompRegSet; Max() is a maximum value operation;

[0127] According to the identification set and the maximum number of background targets, monitoring smoke events; the smoke event is detected when the number of background targets in the identification set is less than the maximum number of background targets;

[0128] In this embodiment, the execution time of this step is 2024 / 06 / 02 / 17:53:02. According to the recognition set CompRegSet and the maximum number of background targets MaxBGsN, a smoke event is detected; the smoke event is detected when the number of background targets RegBGsN in the recognition set CompRegSet=4<the maximum number of background targets MaxBGsN=5; because there is smoke, there is no background target tanker in the compressed image CompImg, resulting in the number of background targets RegBGsN in the recognition set CompRegSet=4; the time of the smoke event is 2024 / 06 / 02 / 17:53:02;

[0129] According to the identification set, monitoring the fire event; the fire event is monitored when at least one target object identifier in the identification set is an identifier of the target object being fire;

[0130] Example 4: According to the identification set CompRegSet, a fire event is detected; the fire event is detected when there is an identifier of a target object (3) in the identification set CompRegSet, which is an identifier of the target object being fire CompRSym(3)=16; the time of the fire event is 2024 / 06 / 04 / 13:35:22;

[0131] According to the identification set, events of people with abnormal behavior are monitored; events of people with abnormal behavior are monitored when at least one target object identifier in the identification symbol set is an identifier of a person with abnormal behavior, or the ASM condition must be met; the identification symbol set is a set of identifiers of target objects including people obtained from the identification set; the ASM condition means that if the abnormal behavior is running or abnormal limb state, the condition that the number of times the abnormal behavior is monitored within the set time length SetTime ≥ NumK must be met; wherein, 2s≤SetTime≤5s; NumK=SetTime×2 times / s; in this embodiment, SetTime=3s.

[0132] Example 5, according to the identification set CompRegSet, an event of a person with abnormal behavior is detected; the event of a person with abnormal behavior is detected when the identifier of the target object (6) in the identification symbol set RegSymbolSet is the identifier RegPISym(1)=7 of the target object being a person with a cigarette in his mouth;

[0133] Embodiment 6, according to the identification set CompRegSet, an event of a person with abnormal behavior is detected; the event of a person with abnormal behavior is detected when there is a target object (8) identifier in the identification symbol set RegSymbolSet, which is the identifier RegPISym(2)=12 of the target object being a person with abnormal limb state, and satisfies the ASM condition; the ASM condition is satisfied because the abnormal behavior is an abnormal limb state, and satisfies the condition that the number of times the abnormal behavior is detected within the set duration of 3s is 7≥NumK=6;

[0134] Example 7, according to the identification set CompRegSet, monitoring the event of a person with abnormal behavior; in this example, no event of a person with abnormal behavior is monitored, because the identifier of the target object (9) in the identification symbol set RegSymbolSet is the identifier RegPISym(3)=6 that the target object is a person;

[0135] The steps of obtaining the recognition symbol set RegSymbolSet from the recognition set CompRegSet are as follows:

[0136] Step 3.1, set the initial value of i to 1, the initial value of t to 0, and set the identification symbol set to an empty set; wherein i is the number of the target object in the identification set, and t is the number of the target object containing a person in the identification set;

[0137] In embodiments 5 to 7, the initial value of i is set to 1, the initial value of t is set to 0, and the identification symbol set RegSymbolSet is set to an empty set;

[0138] Step 3.2: If the identifier of the target object (i) is an identifier of a target object including a person, go to step 3.3; otherwise go to step 3.7; wherein the identifier of the target object (i) is the identifier of the target object numbered i in the identification set CompRegSet;

[0139] Example 5, when i=6, the identifier CompRSym(6) of the target object (6) is the identifier of the target object including a person, and the process goes to step 3.3;

[0140] Example 6, when i=8, the identifier CompRSym(8) of the target object (8) is the identifier of the target object including a person, and the process goes to step 3.3;

[0141] Example 7, when i=9, the identifier CompRSym(9) of the target object (9) is the identifier of the target object including a person, and the process goes to step 3.3;

[0142] Step 3.3, add 1 to the t;

[0143] Embodiment 5, the t is increased by 1; after the addition, t is 1;

[0144] Embodiment 6, the t is increased by 1; after the addition, t is 2;

[0145] Embodiment 7, the t is increased by 1; after the addition, t is 3;

[0146] Step 3.4, obtaining image PI(t) from the image Img according to rectangle CompGRect(i); wherein image PI(t) is an image with width w and height h; the rectangle CompGRect(i) is the minimum bounding rectangle that frames the target object (i) in the recognition set CompRegSet;

[0147] The steps of obtaining the image PI(t) from the image Img according to the rectangle CompGRect(i) are as follows:

[0148] Step 3.4.1, using formula (3) to calculate the left boundary value GSx(t), right boundary value GEx(t), upper boundary value GSy(t), and lower boundary value GEy(t) of the rectangle CompGRect(i) in the rectangle GRect(t) of the image Img,

[0149] GSx(t)=int[CGsx(i)×W / w], GEx(t)=int[CGex(i)×W / w]

[0150] GSy(t)=int[CGsy(i)×H / h], GEy(t)=int[CGey(i)×H / h] (3)

[0151] (3) In the formula, CGsx(i), CGex(i), CGsy(i), and CGey(i) are the left boundary value, right boundary value, upper boundary value, and lower boundary value of the rectangle CompGRect(i), respectively; int[] is an integer operation; W and H are the width and height of the image Img, respectively;

[0152] Embodiment 5, when i=6, t=1, the left boundary value GSx(1), the right boundary value GEx(1), the upper boundary value GSy(1), and the lower boundary value GEy(1) of the rectangle CompGRect(6) in the rectangle GRect(1) of the image Img are calculated using formula (3);

[0153] Embodiment 6, when i=8, t=2, the left boundary value GSx(2), the right boundary value GEx(2), the upper boundary value GSy(2), and the lower boundary value GEy(2) of the rectangle CompGRect(8) in the rectangle GRect(2) of the image Img are calculated using formula (3);

[0154] Embodiment 7, when i=9, t=3, the left boundary value GSx(3), the right boundary value GEx(3), the upper boundary value GSy(3), and the lower boundary value GEy(3) of the rectangle CompGRect(9) in the rectangle GRect(3) of the image Img are calculated using formula (3);

[0155] Step 3.4.2, use formula (4) to calculate the width GRw(t) and height GRh(t) of the rectangle GRect(t),

[0156] GRw(t)=GEx(t)-GSx(t)+1

[0157] GRh(t)=GEy(t)-GSy(t)+1 (4)

[0158] Embodiment 5, when t=1, the width GRw(1) and the height GRh(1) of the rectangle GRect(1) are calculated using formula (4);

[0159] Embodiment 6, when t=2, the width GRw(2) and the height GRh(2) of the rectangle GRect(2) are calculated using formula (4);

[0160] Embodiment 7, at t=3, the width GRw(3) and the height GRh(3) of the rectangle GRect(3) are calculated using formula (4);

[0161] Step 3.4.3: If GRw(t)≤w∧GRh(t)≤h, go to step 3.4.4; otherwise go to step 3.4.5;

[0162] Example 5, when t=1, GRw(1)≤w∧GRh(1)≤h, go to step 3.4.4;

[0163] Example 6, when t=2, GRw(2)≤w, GRh(2)>h, go to step 3.4.5;

[0164] Example 7, when t=3, GRw(3)≤w∧GRh(3)≤h, go to step 3.4.4;

[0165] Step 3.4.4, according to the boundary value of GRect(t), obtain the image PI(t) containing the entire GRect(t) from the image Img; after this step is completed, go to step 3.4.7; the image PI(t) is an image with a width of w and a height of h;

[0166] Embodiment 5, at t=1, according to the boundary value of GRect(1), an image PI(1) including the entire GRect(1) is obtained from the image Img with the center point GRC(1) of GRect(1) as the center; after this step is completed, proceed to step 3.4.7; the image PI(1) is an image with a width of w and a height of h;

[0167] Embodiment 7, at t=3, according to the boundary value of GRect(3), an image PI(3) including the entire GRect(3) is obtained from the image Img with the center point GRC(3) of GRect(3) as the center; after this step is completed, proceed to step 3.4.7; the image PI(3) is an image with a width of w and a height of h;

[0168] Step 3.4.5, according to the boundary value of GRect(t), obtain the image ExpPI(t) containing the entire GRect(t) from the image Img; the image ExpPI(t) is an image with a width of ExpGRw(t) and a height of ExpGRh(t); ExpGRw(t) and ExpGRh(t) are calculated by formula (5),

[0169] ExpGRw(t)=GRw(t), ExpGRh(t)=int[h×GRw(t) / w] if GRw(t) / w≥GRh(t) / h

[0170] ExpGRw(t)=int[w×GRh(t) / h],ExpGRh(t)=GRh(t) else (5)

[0171] Embodiment 6, at t=2, according to the boundary value of GRect(2), an image ExpPI(2) including the entire GRect(2) is obtained from the image Img with the center point GRC(2) of GRect(2) as the center; the image ExpPI(2) is an image with a width of ExpGRw(2) and a height of ExpGRh(2); ExpGRw(2) and ExpGRh(2) are calculated by formula (5), and because

[0172] GRh(2) / h>GRw(2) / w, so we have

[0173] ExpGRw(2)=int[w×GRh(2) / h], ExpGRh(1)=GRh(2) (5)

[0174] Step 3.4.6, compressing the image ExpPI(t) into an image PI(t) with a width of w and a height of h;

[0175] Embodiment 6, at t=2, compressing the image ExpPI(2) into an image PI(2) with a width of w and a height of h;

[0176] Step 3.4.7, end the process of obtaining the image PI(t) from the image Img according to the rectangle CompGRect(i);

[0177] Embodiments 5 to 7, end the process of obtaining the image PI(t) from the image Img according to the rectangle CompGRect(i);

[0178] Step 3.5, using the neural network to recognize the image PI(t) and obtain the identifier RegPISym(t) of the target object (i);

[0179] Example 5, when i=6, t=1, the image PI(1) is recognized by the Yolov8 neural network, and the identifier RegPISym(1)=7 of the target object (6) is obtained (a person with a cigarette in his mouth, as shown in the attached figure) Figure 6 shown);

[0180] Example 6, when i=8, t=2, the image PI(2) is recognized by the Yolov8 neural network, and the identifier RegPISym(2)=12 of the target object (8) is obtained (the angle between the thigh and the upper body of the person is less than 135°, as shown in the attached figure). Figure 6 shown);

[0181] Example 7, when i=9, t=3, the Yolov8 neural network is used to recognize the image PI(3), and the identifier RegPISym(3)=6 of the target object (9) is obtained;

[0182] Step 3.6, add the identifier RegPISym(t) to the identification symbol set RegSymbolSet;

[0183] Embodiment 5, at t=1, adding the identifier RegPISym(1) to the identification symbol set RegSymbolSet;

[0184] Embodiment 6, at t=2, adding the identifier RegPISym(2) to the identification symbol set RegSymbolSet;

[0185] Embodiment 7, at t=3, adding the identifier RegPISym(3) to the identification symbol set RegSymbolSet;

[0186] Step 3.7, add 1 to the i;

[0187] In Example 5 to Example 7, the i is increased by 1;

[0188] Step 3.8, if i>I, go to step 3.9; otherwise go to step 3.2; wherein I is the number of target objects in the identification set;

[0189] In Example 5 to Example 7, if i>I, go to step 3.9; otherwise, go to step 3.2;

[0190] Step 3.9, ending the process of obtaining the recognition symbol set RegSymbolSet from the recognition set CompRegSet;

[0191] Example 7, ending the process of obtaining the identification symbol set RegSymbolSet from the identification set CompRegSet; wherein, RegSymbolSet={RegPISym(t)|t=1,2,3}; RegPISym(1)=7, RegPISym(2)=12, RegPISym(3)=6.

[0192] The event processing includes generating event records, uploading event records, and saving event records. The event processing also includes safety prompts and event alarms.

[0193] In Examples 3 to 6, the steps of event processing are as follows:

[0194] Generate event records; the generating of event records refers to the generating of event record sets by the intelligent analyzer; the elements in the event record set include at least the event name, the time of the event, the event location, the event type, pictures reflecting the event and video recordings reflecting the event;

[0195] In this embodiment, the video recording reflecting the event is a video recording of duration SumTime, and the format of the video is avi; wherein SumTime=T1+EventTL+T2; T1 records the duration before the event occurs, 3s≤T1≤5s; EventTL is the duration of recording the event, 5s≤EventTL≤50s; T2 is the duration recorded after the duration EventTL, 3s≤T2≤5s; in this embodiment, T1=T2=5s; EventTL=50s;

[0196] In Example 3, the intelligent analyzer generates an event record set; the elements in the event record set include the event name being smoke, the time of the event occurring being 2024 / 06 / 02 / 17:53:02; the location of the event occurring being B2 gas station in A1 district of a certain city, the event type being smoke, the picture reflecting the event being a picture of smoke taken at 2024 / 06 / 02 / 17:53:02, and the video recording reflecting the event being a video recording of 60 seconds in length; wherein, T1=T2=5 seconds; EventTL=50 seconds (the duration of the smoke is ≥50 seconds);

[0197] In Example 4, the intelligent analyzer generates an event record set; the elements in the event record set include the event name being fire, the time of the event occurring being 2024 / 06 / 04 / 13:35:22; the location of the event occurring being B3 gas station in A2 district of a certain city, the event type being fire, the picture reflecting the event being a picture of the fire taken at 2024 / 06 / 04 / 13:35:22, and the video recording reflecting the event being a video recording of 18 seconds in length; wherein, T1=T2=5 seconds, and EventTL=8 seconds (the duration of the fire is 8 seconds);

[0198] In Example 5, the intelligent analyzer generates an event record set; the elements in the event record set include the event name being smoking, the time of the event occurring being 2024 / 05 / 05 / 20:37:44; the location of the event occurring being B2 gas station in A2 district of a certain city, the event type being smoking, the picture reflecting the event being a picture of a person smoking taken at 2024 / 05 / 05 / 20:37:44, and the video recording reflecting the event being a video recording of 17 seconds in length; wherein, T1=T2=5 seconds, and EventTL=7 seconds (the duration of smoking is 7 seconds);

[0199] In Example 6, the intelligent analyzer generates an event record set; the elements in the event record set include an event name of abnormal limb state, an event time of 2024 / 05 / 05 / 20:37:44; the event location is a gas station B2 in A2 District of a certain city, an event type of abnormal limb state, a picture reflecting the event is a picture of a person with abnormal limb state taken at 2024 / 05 / 05 / 20:37:44, and a video recording reflecting the event is a video recording with a duration of 15 seconds; wherein, T1=T2=5 seconds, EventTL=5 seconds (duration of abnormal limb state=5 seconds);

[0200] Uploading event records; the uploading event records refers to the intelligent analyzer uploading the event record set to the server through the switch and the system switch;

[0201] In Embodiment 3 to Embodiment 6, the intelligent analyzer uploads the event record set to the server through the switch and the system switch;

[0202] Saving event records: Saving event records means that the server saves the event record set to the storage device of the server;

[0203] In embodiments 3 to 6, the server saves the event record set to the hard disk of the server;

[0204] Safety tips; the safety tips refer to the corresponding safety tips for the events; the corresponding safety tips refer to the events of people with cigarettes in their mouths, people holding cigarettes, people holding lighters, people holding mobile phones, people running, people with abnormal limbs, and the corresponding safety tips are that smoking is strictly prohibited in the refueling area, fireworks are prohibited in the refueling area, fire is strictly prohibited in the refueling area, mobile phones are not allowed in the refueling area, running is not allowed to avoid accidents, and violent actions are not allowed;

[0205] Embodiment 5, issuing a safety reminder that smoking is strictly prohibited in the refueling area; the safety reminder is a safety reminder voice issued by the intelligent analyzer using the speaker connected thereto;

[0206] Embodiment 6, issuing a safety reminder not to do violent actions; the safety reminder is a safety reminder voice issued by the intelligent analyzer using the speaker connected thereto;

[0207] Event alarm; the event alarm means that if the event is a smoke event, a corresponding alarm warning will be issued; if the event is a fire event, a corresponding alarm warning will be issued and the event will be transmitted externally; the event transmission externally means that the server transmits the event record set to the system set location in the external network through the system switch and network equipment.

[0208] Embodiment 3, an alarm warning is issued that there is too much smoke and the user should check it quickly; the alarm warning is an alarm warning voice issued by the smart analyzer using the speaker connected to it.

[0209] Example 4, an alarm warning is issued that a fire has occurred and the user is requested to put it out immediately; the alarm warning is a voice alarm warning issued by the smart analyzer using the speaker connected thereto; the server transmits the event record set to the fire protection center set by the system and the management center of the monitoring system in the external network through the system switch and network equipment.

[0210] The working principle of the above technical solution is as follows: the intelligent analyzer captures a frame of image Img from the video image output by the high-resolution camera; compresses the image Img into a compressed image CompImg; identifies the target objects in the image (such as identifying target (1), target (2), ..., target (7)) from the compressed image CompImg based on the Yolov8 neural network, obtains an identification set, and then monitors events according to the identification set; wherein, monitoring of smoke events is performed based on whether the number of identified background targets is less than the maximum number of background targets (when there is no smoke); monitoring of fire events is performed based on whether the identifier of the target object in the identification set is a fire identifier, and fire can be reliably detected when the width ≥56mm and the height ≥42mm; when a target object containing a person is detected, first, based on the minimum circumscribed rectangle of the target object containing the person in the compressed image, obtain an image PI(t) containing the entire target object with a width of w and a height of h from the original high-resolution image Img, and then use Yolo The Yolov8 neural network recognizes this image PI(t) and obtains a set of identification symbols, and then determines whether there is an event based on whether the identifier of the target object in the identification symbol set is an identifier of a person with abnormal behavior. The reason for this is that cigarettes, lighters, mobile phones, and human limbs are relatively small in size (for example, the minimum diameter of a cigarette is only 7.5mm), and the Yolov8 neural network recognizes compressed images. It is difficult for the Yolov8 neural network to recognize small objects such as cigarettes, lighters, mobile phones, and human limbs in compressed images. When the present technical solution uses the Yolov8 neural network to recognize the image PI(t), because the image PI(t) is an image captured from a high-resolution (such as 4472×3354) original image, this resolution can accurately monitor small objects such as cigarettes, lighters, mobile phones, and human limbs within the range monitored by the camera. Therefore, the present technical solution can reliably monitor events of people with abnormal behavior based on the identified cigarettes, lighters, mobile phones, and human limbs within the range monitored by the camera (such as a width of 8.38m and a height of 6.25m).

[0211] The beneficial effects of the above technical solution are: using the Yolov8 neural network to identify the target object can accurately monitor events with smoke and fire; the width w and height h of the image processed by the Yolov8 neural network in this technical solution are both equal to 608, such w and h can reliably identify fire and other objects with a width ≥ 56mm and a height ≥ 42mm within the range monitored by the camera, which can identify smaller objects than the existing technical solution, and the reliability and accuracy of recognition are also higher; when monitoring a person, the technical solution intercepts the w*h image PI(t) (w*h The image is an image with a width of w and a height of h), and then the Yolov8 neural network is used to recognize the image PI(t), and a recognition symbol set is obtained, and then the event of a person with abnormal behavior is monitored according to the recognition symbol set. The present technical solution not only uses the existing technical solution to monitor with the posture of the human hand, but also mainly monitors by identifying cigarettes, lighters, mobile phones, and human limbs, and its monitoring is carried out using the resolution of the original image; it can be seen that the present technical solution can monitor more target objects such as cigarettes, lighters, mobile phones, and human limbs than the existing technical solution, and the monitoring is more extensive, the recognition reliability is higher, and the recognition accuracy and practicality are better; therefore, it is particularly suitable for gas stations and other safety places. In addition, compared with the prior art method of using the w*h image to cover the original image to find the target, the present technical solution has higher recognition efficiency, better recognition accuracy and more complete recognized targets. This is because the prior art usually scans the w*h image in an overlapping manner in the X and Y directions in order not to segment the target (e.g. the w*h image on the right overlaps 1 / 2 of the w*h image on the left in the X direction, and the w*h image below overlaps 1 / 2 of the w*h image above in the Y direction), which not only increases the number of scans, but also reduces the efficiency and real-time performance of recognition, and it is also impossible to guarantee that an image containing the entire person can be obtained (e.g. the image at the edge of the overlapping area Another defect of the prior art that uses a w*h image to cover the original image to find the target is that if the target is close to the camera, the width of the target may exceed w, or the height of the target may exceed h, so the acquired w*h image may not obtain the complete target, resulting in low reliability, completeness and accuracy of recognition; the method of this technical solution to deal with this situation is to first extract an image containing the complete target from the image (such as ExpPI(t)), and then compress the image containing the complete target (such as ExpPI(t)) into a w*h image PI(t), and then use the Yolov8 neural network to recognize this w*h image PI(t);This not only ensures the integrity, accuracy and reliability of recognition, but also greatly improves the efficiency of processing and recognition, making the method more practical, reliable, accurate, real-time and more extensive. ;

[0212] In Example 1 and Example 2, the steps of generating the neural network are as follows:

[0213] In this embodiment, the process of generating a neural network is as shown in the attached Figure 5 As shown;

[0214] Step 5.1, set the number of acquired pictures PNum, and set the processing set to an empty set; wherein PNum is a positive integer;

[0215] Example 1, set PNum=12000; set the processing set ProcSet to an empty set, and set the following sets to empty sets: atlas set ImgSet, preprocessing set PreProcSet, compression set CompSet, network atlas set NetSet, set target atlas set AppImgSet, set target processing set AppProcSet;

[0216] Step 5.2, obtaining pictures; the pictures are pictures containing the target object and the number of obtained pictures is ≥ PNum;

[0217] In this embodiment, a picture is obtained and saved to the atlas ImgSet; ImgSet={Pm(j)|j=1,2,...,J2}; wherein Pm(j) is a picture; j is a number formed in the order of the time when the picture is added to the atlas ImgSet; J2 is the total number of pictures in the atlas ImgSet, and J2 is a positive integer; J2=SumJ+NA; SumJ is the number of pictures obtained from the video recording, and NA is the number of pictures obtained from the external network;

[0218] Example 1, J2=15280 (>PNum=12000); wherein, SumJ=13280, NA=2000;

[0219] Example 2, J2=20260 (>PNum=15600); wherein, SumJ=17260, NA=3000;

[0220] Step 5.3, using the image to obtain a processing set;

[0221] In this embodiment, the processing set ProcSet is obtained using the pictures in the atlas ImgSet; ProcSet={CompP(aj), CompPFile(aj)|aj=1,2,...,J}; where CompP(aj) is a picture with a width of w and a height of h; CompPFile(aj) is a labeling file of the picture CompP(aj); J is the number of pictures in the processing set ProcSet, J=J2+PV; PV is the number of pictures in the processing set of the set target object;

[0222] Example 1, J=J2+PV=22870; wherein, J2=15280, PV=7590;

[0223] Example 2, J=J2+PV=29890; wherein, J2=20260, PV=9630;

[0224] Step 5.4, splitting the processing set into a training set and a test set;

[0225] In this embodiment, the processing set ProcSet is split into a training set TrainSet and a test set TestSet; wherein TrainSet={CompP(q), CompPFile(q)|q=1,2,...,Q}, Q=0.6×J; TestSet={CompP(s), CompPFile(s)|s=1,2,...,S}, S=0.4×J;

[0226] Step 5.5, training the neural network model with the training set; training the neural network model with the training set means that the training neural network model program in the GPU server uses each group of pictures CompP(q) and the annotation file CompPFile(q) in the training set TrainSet to train the Yolov8 neural network model, q=1,2,...,Q;

[0227] Step 5.6, using the test set to test the neural network model and counting the test data of each target object; the test data of the target object refers to the number of target object tests and the number of correct tests of the target object;

[0228] The steps of testing the neural network model with the test set and counting the test data of each target object are as follows:

[0229] Step 5.6.1, set the initial value of s to 1, set TestPn to the number of images in the test set TestSet, set the initial value of each SumTN(k) to 0 and set the initial value of each SumCN(k) to 0, k=1,2,...,K; where s is the number of images and annotation files in a set of images and annotation files in the test set; SumTN(k) is the number of tests for target object k; SumCN(k) is the number of correct tests for target object k;

[0230] Example 1, set s=1, set TestPn=9148 (=0.4×J), set each SumTN(k)=0 and set each SumCN(k)=0, k=1,2,...,16;

[0231] Example 2, set s=1, set TestPn=11956 (=0.4×J), set each SumTN(k)=0 and set each SumCN(k)=0, k=1,2,...,16;

[0232] Step 5.6.2, the test neural network model program in the GPU server uses the image CompP(s) in the test set TestSet and the annotation file CompPFile(s) to test the Yolov8 neural network model, and counts each TestNum(k) and each CorrectNum(k), k=1,2,...,K; wherein, TestNum(k) is the number of target objects k in the annotation file CompPFile(s); CorrectNum(k) is the number of target objects k tested correctly; a target object (m) in the annotation file CompPFile(s) that is target object k is tested correctly means that the Yolov8 neural network model's recognition of the target object (m) in the test satisfies the following three conditions at the same time: ① Identify the target object (m) from the image CompP(s) ;②The rectangle TestRect(m) of the identified target object (m) satisfies the condition [Pn(TestRect(m)∩CompRect(m)) / Pn(CompRect(m))]×100%≥MinSv; where Pn(AR) is the number of pixels in the area AR; 80%≤MinSv≤100%; CompRect(m) is the minimum enclosing rectangle that frames the target object (m) in the annotation file CompPFile(s);③The identifier TestSym(m) of the target object (m)=CompSym(m); TestSym(m) is the marker of the target object (m) identified by the Yolov8 neural network model; CompSym(m) is the marker of the target object (m) in the annotation file CompPFile(s)=the marker of the target object k;

[0233] In this embodiment, when s=1, the test neural network model program in the GPU server uses the image CompP(1) in the test set TestSet and the annotation file CompPFile(1) to test the Yolov8 neural network model, and statistics are obtained as follows: TestNum(k)=1, CorrectNum(k)=1, k=1, 2, ..., 5; TestNum(6)=2, CorrectNum(6)=2; TestNum(k)=0, CorrectNum(k)=0, k=7, 8, ..., 16; the target object (1) in the annotation file CompPFile(1) is the target object 6. Correct test means that the Yolov8 neural network model meets the following three conditions for the recognition of the target object (1) in the test: ① the target object (1) is recognized from the image CompP(1); ② the recognized target object (1) is recognized; The rectangle TestRect(1) at the position of the target object (1) satisfies the condition [Pn(TestRect(1)∩CompRect(1)) / Pn(CompRect(1))]×100%≥85%; wherein, MinSv=85%; CompRect(1) is the minimum enclosing rectangle that frames the target object (1) in the annotation file CompPFile(1); ③ The identifier of the target object (1) TestSym(1)=CompSym(1); TestSym(1) is the marker of the target object (1) identified by the Yolov8 neural network model; CompSym(1) is the marker of the target object (1) in the annotation file CompPFile(1) = the marker of the target object 6 = 6; similarly, it can be obtained that another target object (2) in the annotation file CompPFile(1) is the target object 6 and is also tested correctly;

[0234] Step 5.6.3, accumulate each SumCN(k) and accumulate each SumTN(k), k=1,2,...,K;

[0235] In Example 1 and Example 2, each SumCN(k)=SumCN(k)+CorrectNum(k) is accumulated, and each SumTN(k)=SumTN(k)+TestNum(k) is accumulated, k=1,2,...,16; wherein SumCN(k) and SumTN(k) on the left side of the formula are both accumulated values, and SumCN(k) and SumTN(k) on the right side of the formula are both values ​​before accumulation;

[0236] Embodiment 1, when s=1, SumCN(k)=1, SumTN(k)=1, k=1, 2, ..., 5; SumCN(6)=2, SumTN(6)=2; SumCN(k)=0, SumTN(k)=0, k=7, 8, ..., 16;

[0237] Step 5.6.4, add 1 to the s;

[0238] Step 5.6.5: If s>TestPn, go to step 5.6.6; otherwise go to step 5.6.2;

[0239] Step 5.6.6, end the process of testing the neural network model with the test set and counting the test data of each target object;

[0240] In this embodiment, the process of testing the Yolov8 neural network model with the test set and counting the test data of each target object is completed;

[0241] Step 5.7, calculating the test accuracy of each target object according to the test data of each target object;

[0242] In Example 1 and Example 2, the test accuracy rate Av(k) of each target object k is calculated according to the test data of each target object k, k=1,2,...,16; wherein Av(k)=[SumCN(k) / SumTN(k)]×100%;

[0243] Step 5.8: If the test accuracy rate Av(k) of each target object k is ≥ SetAv, k=1,2,...,K, then go to step 5.10; otherwise, go to step 5.9; wherein SetAv is the set accuracy rate, 85%≤SetAv≤99%, in this embodiment, SetAv=98%;

[0244] Example 1, the test accuracy of target object 7 Av(7)=90%<98%, go to step 5.9;

[0245] Example 2, the test accuracy Av(k) of each target object k is ≥98%, k=1, 2, ..., 16, go to step 5.10;

[0246] Step 5.9, increase the PNum value, and then go to step 5.2; increasing the PNum value means that the PNum after increase = the PNum before increase × AddScale; wherein, 1.2≤AddScale≤2.0;

[0247] Example 1, increase the PNum value, and after the increase, go to step 5.2; after the increase, PNum=15600; wherein, AddScale=1.3;

[0248] Step 5.10, compiling the neural network model into a neural network; the neural network is a neural network that can recognize the target object;

[0249] In embodiment 2, the GPU server compiles the Yolov8 neural network model into a Yolov8 neural network using a compiling software; the Yolov8 neural network is a Yolov8 neural network that can identify the target object;

[0250] Step 5.11, write the neural network into each intelligent analyzer in the system.

[0251] In Example 2, the server takes out the Yolov8 neural network in the GPU server through the system switch, and writes the Yolov8 neural network into each intelligent analyzer in the system through the system switch and the switch in each monitoring subsystem of the 24 monitoring subsystems; the time for completion of this step is 2024 / 04 / 28 / 17:50:03.

[0252] In Example 1 and Example 2, the steps of obtaining the image are as follows:

[0253] Step 6.1, obtaining pictures from the video recording; wherein the video recording is the video recording recorded by the network video recorder; the pictures are pictures containing the target object and are different from each other;

[0254] The steps to get pictures from video recordings are as follows:

[0255] Step 6.1.1, obtaining video; obtaining video means obtaining D1 video recordings containing the target object; D1 is a positive integer;

[0256] In this embodiment, the server uses the video downloading software to download D1 segments of video recordings d1 containing the target object in different time periods from each of the three network video recorders connected thereto, where d1=StrDx,StrDx+1,...,D1; wherein d1 is the number of the video recording segments generated in the order of the time when the video recording segments are downloaded to the server; D1 is the number of the downloaded video recording segments; and the duration of each video recording segment is 1 hour;

[0257] Embodiment 1, StrDx=1, D1=100;

[0258] Embodiment 2, StrDx=101, D1=130;

[0259] Step 6.1.2, get the picture; get the picture means to capture the picture from each video recording d1 and add the captured picture to the temporary storage set TmpSet(d), d1=StrDx,StrDx+1,...,D1, d=StrD,StrD+1,...,D, and d1 is equal to StrDx,

[0260] StrDx+1,...,D1, the corresponding d is equal to StrD, StrD+1,...,D respectively; TmpSet(d)={Pt(d,u)|u=1,2,...,U(d)}; u is the number of pictures in the temporary set TmpSet(d) in the order of the time they are added to the temporary set TmpSet(d); U(d) is the number of pictures in the temporary set TmpSet(d), and U(d) is a positive integer; the resolution of the picture Pt(d,u) is the resolution of the video recording; the time interval for the screenshot software to capture pictures from the video recording d1 is TimeSpace seconds; 10s≤TimeSpace≤30s; in this embodiment, TimeSpace=20s;

[0261] Embodiment 1, StrDx=1, D1=100; StrD=1, D=100;

[0262] Embodiment 2, StrDx=101, D1=130; StrD=101, D=130;

[0263] Step 6.1.3, filter the image;

[0264] In this embodiment, the steps of filtering pictures are as follows:

[0265] Step 6.1.3.1, delete the same pictures; deleting the same pictures means using the picture management software ACDSee to delete the same pictures in each temporary set TmpSet(d), d=StrD,StrD+1,...,D; the pictures Pt(d,u1) and Pt(d,u2) in the temporary set TmpSet(d) are the same means that each target object in the two pictures is the same and the number of targets is the same, and each PArea(d,u1,k)∩PArea a(d,u2,k) are not empty sets; where u1≠u2; PArea(d,u1,k) and PArea(d,u2,k) are the area where the target object k in the image Pt(d,u1) is located and the area where the same target object k in the image Pt(d,u2) is located, respectively, and the operation of PArea(d,u1,k)∩PArea(d,u2,k) is performed when the images Pt(d,u1) and Pt(d,u2) are in the same image coordinate system;

[0266] Step 6.1.3.2, save the pictures in the temporary atlas to the atlas; saving the pictures in the temporary atlas to the atlas means adding each picture Pt(d,u) in each temporary set TmpSet(d) to the atlas ImgSet with the name Pm(j), d=StrD,StrD+1,...,D, u=1,2,...,U(d), j=1,2,...,SumJ; where j is the number generated in the order of the time when the pictures are added to the atlas ImgSet; SumJ is the number of pictures in ImgSet; after this step is completed, the atlas ImgSet={Pm(j)|j=1,2,...,SumJ};

[0267] Example 1, StrD=1, D=100, SumJ=13280 (≥PNum=12000);

[0268] Example 2, StrD=101, D=130, SumJ=17260 (≥PNum=15600);

[0269] Step 6.2: Obtain a picture containing the set target object from the external network.

[0270] In this embodiment, the steps of obtaining a picture containing a set target object from an external network are as follows:

[0271] Step 6.2.1, the server obtains pictures containing set target objects from the external network through the system switch and network equipment, and adds these pictures containing set target objects to the network atlas NetSet; at the end of this step, NetSet={Pnet(a)|a=1,2,...,NA}; wherein a is the number of pictures formed in the order of the time when the pictures are added to the network atlas NetSet; NA is the number of pictures in the network atlas NetSet, and NA is a positive integer;

[0272] Example 1, NA=2000;

[0273] Example 2, NA=3000;

[0274] Step 6.2.2, add each picture in the network atlas that has not been added to the atlas to the atlas; adding each picture in the network atlas that has not been added to the atlas to the atlas means adding each picture Pnet(a) in the network atlas NetSet that has not been added to the atlas ImgSet to the atlas ImgSet with the name Pm(j), a=Sa, Sa+1,..., NA, j=SumJ+Sa, SumJ+Sa+1,..., SumJ+NA, and a is equal to Sa, Sa+1,..., NA in sequence, and the corresponding j is equal to SumJ+Sa, SumJ+Sa+1,..., SumJ+NA in sequence;

[0275] Example 1, Sa=1, NA=2000, SumJ=13280.

[0276] Example 2, Sa=2001, NA=3000, SumJ=17260.

[0277] In Example 1 and Example 2, the steps of obtaining the processing set using the image are as follows:

[0278] Step 7.1, labeling processing; labeling processing refers to using automatic labeling software to automatically label each unlabeled picture Pm(j) in the picture, and obtain the labeling file LabPFile(pj) of the picture, j=SJ,SJ+1,...,J2, pj=SP,SP+1,...,J2, and j is equal to SJ,SJ+1,...,J2 in sequence, and the corresponding pj is equal to SP,SP+1,...,J2 in sequence; then the picture Pm(j) is added to the preprocessing set PreProcSet with the name P(pj), and the labeling file LabPFile(pj) is added to the Preprocessing set PreProcSet; wherein the annotation file LabPFile(pj) at least includes the file name, the file save path, the width and height of the image Pm(j) and the marking value of the color image, the number of identified targets Mp(pj), and the following data of each target (m): a rectangle Rect(m) marking the position of the target (m), a marker LabSym(m) of the target (m), m=1,2,...,Mp(pj); wherein the rectangle marking the position of the target (m) is a rectangle represented by four boundary values; m is a number formed in increasing order of the center points of the rectangle Rect(m);

[0279] Example 1, SJ=1, SP=1, J2=15280;

[0280] Example 2, SJ=15281, SP=15281, J2=20260;

[0281] At the end of this step, the preprocessing set PreProcSet={P(pj),LabPFile(pj)|pj=1,2,...,J2}; wherein pj is the number of the picture P(pj) and the annotation file LabPFile(pj);

[0282] Step 7.2, correction processing; correction processing refers to correcting an incorrect annotation file into a correct annotation file using a modification annotation software, and saving the corrected annotation file LabPFile(pj) to the original annotation file in the preprocessing set PreProcSet; the correct annotation file refers to the number of targets Mp(pj) in the annotation file LabPFile(pj) = the number of targets in the image P(pj), and the following data of each target (m) in the annotation file LabPFile(pj): the minimum circumscribed rectangle Rect(m) that frames the target (m), the marker LabSym(m) of the target (m) is the correct marker, m=1,2,...,Mp(pj);

[0283] Step 7.3, compression processing; compression processing refers to compressing the uncompressed picture P(pj) in the picture to obtain the compressed picture CompG(cj) of the picture, and obtaining the annotation file CompGFile(cj) of the compressed picture CompG(cj) according to the annotation file LabPFile(pj) of the picture P(pj), pj=SP,SP+1,...,J2, cj=SC,SC+1,...,J2, and pj is equal to SP,SP+1,...,J2 in sequence, and the corresponding cj is equal to SC,SC+1,...,J2 in sequence; then the compressed picture CompG(cj) and the annotation file LabPFile(pj) are added to the compression set CompSet; wherein the annotation file CompGFile(cj) at least includes the file name, the file save path, the width and The height and the marking value of the color image, the number of targets Mc(cj) (=Mp(pj)) in the image CompG(cj), and the following data of each target (m): the rectangle CompRect(m) marking the position of the target (m), the marker CompSym(m) (=LabSym(m)) of the target (m), m=1,2,...,Mc(cj); the X-direction boundary value of CompRect(m) is the boundary value obtained by compressing the X-direction boundary value of Rect(m) according to the compression ratio w / W(pj), and the Y-direction boundary value of CompRect(m) is the boundary value obtained by compressing the Y-direction boundary value of Rect(m) according to the compression ratio h / H(pj); the X-direction boundary value refers to the left boundary value and the right boundary value; the Y-direction boundary value refers to the upper boundary value and the lower boundary value; W(pj) and H(pj) are the width and height of the image P(pj) respectively;

[0284] Example 1, SP=1, J2=15280, SC=1;

[0285] Example 2, SP=15281, J2=20260, SC=15281;

[0286] After this step is completed, the compression set CompSet={CompG(cj),CompGFile(cj)|cj=1,2,...,J2};

[0287] Step 7.4, using each set of pictures and annotation files to obtain a set target picture and an annotation file of the set target picture; the set of pictures and annotation files refers to a picture in the set and an annotation file of the picture; the picture in each set of pictures refers to a picture in the pictures; the set target picture is a picture that contains at least one set target and has a width of w and a height of h;

[0288] In this embodiment, each set of images and annotation files in the preprocessing set are used to obtain the set target image and the annotation file of the set target image, and the obtained set target image and the annotation file of the set target image are added to the set target processing set AppProcSet;

[0289] Step 7.5: Use each group of compressed images and annotation files and each group of set target object images and annotation files to form a processing set.

[0290] In this embodiment, the steps of using each group of compressed images and annotation files in the compression set CompSet and each group of set target object images and annotation files in the set target object processing set AppProcSet to form the processing set ProcSet are as follows:

[0291] Each image CompG(cj) in the compression set CompSet is added to the processing set ProcSet with the name CompP(aj), and the annotation file CompGFile(cj) of the image is added to the processing set ProcSet with the name CompPFile(aj), cj=SC, SC+1,..., J2, aj=AStrJ, AStrJ+1,..., AEndJ, and cj is equal to SC, SC+1,..., J2 in sequence, and the corresponding aj is equal to AStrJ, AStrJ+1,..., AEndJ in sequence;

[0292] Example 1, SC=1, J2=15280, AStrJ=1, AEndJ=15280;

[0293] Example 2, SC=15281, J2=20260, AStrJ=22871, AEndJ=27850;

[0294] Each image ImgG(pv) in the set target object processing set AppProcSet is added to the processing set ProcSet with the name CompP(aj), and the annotation file LabIGFile(pv) of the image ImgG(pv) in the set target object processing set AppProcSet is added to the processing set ProcSet with the name CompPFile(aj), pv=StrPv,StrPv+1,...,PV, aj=AEndJ+1,AEndJ+2,...,J, and pv is equal to StrPv,StrPv+1,...,PV in sequence, and the corresponding aj is equal to AEndJ+1,AEndJ+2,...,J in sequence;

[0295] Example 1, StrPv=1, PV=7590, AEndJ+1=15281, J=22870;

[0296] Example 2, StrPv=7591, PV=9630, AEndJ+1=27851, J=29890;

[0297] After this step is completed, the processing set ProcSet={CompP(aj), CompPFile(aj)|aj=1,2,...,J}; wherein J is the number of images in the processing set ProcSet.

[0298] The steps of obtaining the set target object image and the set target object image's annotation file using each set of images and annotation files are as follows:

[0299] In Example 1 and Example 2, the steps of obtaining the set target object image and the set target object image's annotation file using each set of images and annotation files in the preprocessing set are as follows:

[0300] Step 8.1, use each set of images and annotation files to obtain the set target object images;

[0301] In Example 1 and Example 2, the steps of obtaining the set target object image using each group of images and the annotation file in the preprocessing set are as follows:

[0302] Step 8.1.1, set the initial value of pj to PStrJ, set the initial value of v to StrV; pj is the number of the image that has not been taken in the preprocessing set; SP is the starting number of the image that has not been taken in the preprocessing set; v is the number of the image in the set target image set; StrV is the starting number of the image in the set target image set; the image that has not been taken means that the set target image has not been obtained;

[0303] Example 1, set pj=SP=1, set v=StrV=0;

[0304] Example 2, set pj=SP=15281, set v=StrV=15280;

[0305] Step 8.1.2, obtaining a set target object image from the image P(pj) and the annotation file LabPFile(pj); the image P(pj) and the annotation file LabPFile(pj) are both the image and the annotation file in the preprocessing set;

[0306] The steps to obtain the set target image from the image P(pj) and the annotation file LabPFile(pj) are as follows:

[0307] Step 8.1.2.1, set the initial value of m to 1; wherein m is the number of the target object in the annotation file LabPFile(pj) in the preprocessing set;

[0308] In this embodiment, when pj=2, m=1 is set;

[0309] Step 8.1.2.2: If the identifier LabSym(m) of the target object(m) is the identifier of the set target object, go to step 8.1.2.3; otherwise, go to step 8.1.2.7;

[0310] In this embodiment, when pj=2, m=1, the identifier LabSym(1) of the target object (1) is the identifier of the target object 13, the cigarette, and the process goes to step 8.1.2.3;

[0311] In this embodiment, when pj=2, m=2, the identifier LabSym(2) of the target object (2) is not the identifier of any set target object, and the process goes to step 8.1.2.7;

[0312] In this embodiment, when pj=2, m=3, the identifier LabSym(3) of the target object (3) is the identifier of the target object 6, and the process goes to step 8.1.2.3;

[0313] Step 8.1.2.3, add 1 to the v;

[0314] In this embodiment, when pj=2 and m=1, 1 is added to the v; after the addition, v=1;

[0315] In this embodiment, when pj=2 and m=3, 1 is added to the v; after the addition, v=2;

[0316] Step 8.1.2.4, extracting the minimum circumscribed rectangle Rect(m) that frames the target object (m) from the annotation file LabPFile(pj);

[0317] In this embodiment, when pj=2 and m=1, the minimum circumscribed rectangle Rect(1) enclosing the target object (1) is taken out from the annotation file LabPFile(2);

[0318] In this embodiment, when pj=2 and m=3, the minimum circumscribed rectangle Rect(3) enclosing the target object (3) is taken out from the annotation file LabPFile(2);

[0319] Step 8.1.2.5, according to the minimum circumscribed rectangle Rect(m), obtain the image ImgP(v) from the image P(pj); wherein the image ImgP(v) is an image that contains the entire rectangle Rect(m) and has a width of w and a height of h;

[0320] According to the minimum enclosing rectangle Rect(m), the steps of obtaining the image ImgP(v) from the image P(pj) are as follows:

[0321] Step 8.1.2.5.1, use formula (6) to calculate the width Rw(m) and height Rh(w) of the minimum circumscribed rectangle Rect(m),

[0322] Rw(m)=REx(m)-RSx(m)+1

[0323] Rh(m)=REy(m)-RSy(m)+1 (6)

[0324] (6) In the formula, RSx(m), REx(m), RSy(m), and REy(m) are the left boundary value, right boundary value, upper boundary value, and lower boundary value of the rectangle Rect(m), respectively;

[0325] In this embodiment, when m=1, the width Rw(1) and height Rh(1) of the minimum circumscribed rectangle Rect(1) are calculated using formula (6);

[0326] In this embodiment, when m=3, the width Rw(3) and height Rh(3) of the minimum circumscribed rectangle Rect(3) are calculated using formula (6);

[0327] Step 8.1.2.5.2: If Rw(m)≤w∧Rh(m)≤h, go to step 8.1.2.5.3; otherwise go to step 8.1.2.5.4;

[0328] In this embodiment, when m=1, Rw(1)≤w∧Rh(1)≤h, go to step 8.1.2.5.3;

[0329] In this embodiment, when m=3, Rw(3)>w, Rh(3)<h, go to step 8.1.2.5.4;

[0330] Step 8.1.2.5.3, according to the boundary value of Rect(m), obtain the w*h area containing the entire rectangle Rect(m) from the image P(pj), and save the image of this w*h area as the image ImgP(v); after this step is completed, go to step 8.1.2.5.5; wherein, the w*h area is an area with a width of w and a height of h;

[0331] In this embodiment, when m=1, pj=2, v=1, the w*h area including the entire rectangle Rect(1) is obtained from the image P(2) according to the boundary value of Rect(1), and the image of this w*h area is saved as the image ImgP(1); after this step is completed, go to step 8.1.2.5.5;

[0332] Step 8.1.2.5.4, according to the boundary value of Rect(m), obtain the ERw*ERh area containing the entire rectangle Rect(m) from the image P(pj), compress the image of the ERw*ERh area into a w*h image, and then save the w*h image as the image ImgP(v); wherein the ERw*ERh area is an area with a width of ERw and a height of ERh; ERw and ERh are calculated by formula (7),

[0333] ERw=Rw(m), ERh=int[h×Rw(m) / w] if Rw(m) / w≥ Rh(m) / h

[0334] ERw=int[w× Rh(m) / h],ERh=Rh(m) else (7)

[0335] In this embodiment, when m=3, pj=2, and v=2, the ERw*ERh region including the entire rectangle Rect(3) is obtained from the image P(2) according to the boundary value of Rect(3), and the image of the ERw*ERh region is compressed into a w*h image, and then the w*h image is saved as the image ImgP(2); wherein the ERw*ERh region is a region with a width of ERw and a height of ERh; ERw and ERh are calculated by formula (7) (because Rw(3) / w>1.0, Rh(3) / h<1.0), so,

[0336] ERw=Rw(3), ERh=int[h× Rw(3) / w] (7)

[0337] Step 8.1.2.5.5, end the process of obtaining the image ImgP(v) from the image P(pj) according to the minimum circumscribed rectangle Rect(m);

[0338] Step 8.1.2.6, adding the image ImgP(v) to the set target object atlas AppImgSet;

[0339] In this embodiment, when v=1 and v=2, the pictures ImgP(1) and ImgP(2) are added to the set target picture set AppImgSet in sequence;

[0340] Step 8.1.2.7, add 1 to the m;

[0341] Step 8.1.2.8, if m>Mp(pj), go to step 8.1.2.9, otherwise go to step 8.1.2.2; where Mp(pj) is the number of targets in the annotation file LabPFile(pj);

[0342] In this embodiment, when pj=2, if m>Mp(2), go to step 8.1.2.9, otherwise go to step 8.1.2.2;

[0343] Step 8.1.2.9, end the process of obtaining the set target object image from the image P(pj) and the annotation file LabPFile(pj);

[0344] Step 8.1.3, add 1 to the pj;

[0345] Step 8.1.4: If pj>J2, go to step 8.1.5; otherwise go to step 8.1.2;

[0346] Step 8.1.5, ending the process of obtaining the set target object image using each set of images and the annotation file;

[0347] In this embodiment, when the processing is finished, the target object atlas AppImgSet={ImgP(v)|v=1,2,...,V} is set;

[0348] Embodiment 1, V=7590;

[0349] Embodiment 2, V=9630;

[0350] Step 8.2, using the set target object image to obtain a labeling file of the set target object image;

[0351] In this embodiment, the steps of obtaining the annotation file of the set target object picture using the set target object picture in the set target object picture set AppImgSet are as follows:

[0352] Step 8.2.1, use automatic labeling software to automatically label each unlabeled set target object image ImgP(v) in the set target object atlas AppImgSet, and obtain the labeling file LabIGFile(pv) of the set target object image, v=StrV,StrV+1,...,V, pv=PStrV,PStrV+1,...,PV, and v is equal to StrV,StrV+1,...,V in sequence, and the corresponding pv is equal to PStrV,PStrV+1,...,PV in sequence; then add the picture ImgP(v) to the set target object processing set AppProcSet with the name ImgG(pv), and add the labeling file LabIGFile(pv) to the set target object processing set AppProcSet; after this step is completed, AppProcSet={ImgG(pv),LabIGFile(pv)|pv=1,2,...,PV};

[0353] Example 1, StrV=1, V=7590, PStrV=1, PV=7590;

[0354] Example 2, StrV=7591, V=9630, PStrV=7591, PV=9630;

[0355] Step 8.2.2, use the label modification software LabelImg to correct the incorrect label file of the set target object image in the set target object processing set AppProcSet to a correct label file, and save the corrected label file to the original label file in the set target object processing set AppProcSet.

[0356] The beneficial effect of the above technical solution is that the video images containing the target object are recorded in different time periods and under different environmental conditions, and then the screenshot software is used to capture pictures containing the target object in different environments and different lighting conditions from the video recording, and more pictures containing cigarettes, lighters, mobile phones, fire, and people with abnormal behavior are obtained from the external network; after a sufficient number of pictures are obtained, each picture is automatically annotated to obtain the annotation file of the picture; then the incorrect annotation file is corrected to the correct annotation file by modifying the annotation software; then each picture (in the pre-processing set) is compressed into a compressed picture with a width of w and a height of h, and the annotation file of the compressed picture is obtained according to the annotation file of the picture; on the other hand, the technical solution also The set target object images are obtained from each group of images and annotation files in the (pre-processing set), and then the annotation files of these set target object images are obtained through annotation processing and correction processing; then, each group of compressed images and annotation files in the (compression set) and each group of set target object images and annotation files in the (set target object processing set) are used to form a processing set; then the processing set is split into a training set and a test set; thereafter, the neural network model is trained with the training set, the neural network model is tested with the test set and the test data of each target object is counted; after the accuracy of each target object test is ≥ the set accuracy rate, the neural network model is compiled into a neural network for identifying the target object using compilation software, and then the neural network for identifying the target object is written into each intelligent analyzer in the system. The technical solution not only obtains pictures containing the target objects in actual scenes, different environments and different lighting conditions, but also obtains pictures containing cigarettes, lighters, mobile phones and people from pictures (in the preprocessing set) and annotation files, and trains the neural network with these pictures of actual scenes, rich content and clear details, which greatly improves the accuracy and reliability of the neural network in identifying the target objects (especially cigarettes, lighters, mobile phones and human limbs), thereby greatly improving the practicality, stability and accuracy of the event monitoring method, and greatly improving the extensiveness; in addition, the target object pictures are automatically obtained from each group of pictures and annotation files (in the preprocessing set), which not only enriches the content of the pictures, but also greatly reduces the workload of obtaining the set target object pictures, and also greatly improves the accuracy and consistency of obtaining the set target object pictures; the neural network can learn more extensive, richer, more detailed, more complete and more practical features, thereby improving the generalization ability and robustness of the neural network, so that the neural network has stronger recognition ability, wider recognition content, more accurate recognition rate and faster processing efficiency.

[0357] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A gas station safety monitoring method based on vision, wherein the method is implemented by a gas station safety monitoring system, the gas station safety monitoring system includes N monitoring subsystems, a server, a GPU server, T network video recorders, a system switch, a network device and a system power supply, the monitoring subsystem includes an intelligent analyzer, a switch, a camera, a speaker and a power supply, characterized in that: The steps of the method described are as follows: Step A, generating a neural network; the generating neural network includes acquiring a picture, using the picture to acquire a processing set, and using the processing set to train a neural network model to obtain a neural network capable of identifying a target object; wherein the picture is a picture containing a target object and the number of acquired pictures is ≥ PNum; PNum is a positive integer; acquiring a processing set using the picture includes labeling processing, correction processing, and compression processing, and obtaining a processing set consisting of a compressed picture and a labeling file of the compressed picture and a set target object picture and a labeling file of the set target object picture; the target object includes background targets and set targets; the background targets include at least a refueling island, a refueling machine, a column, a crash barrier, and a wall; the set targets include at least targets including people, cigarettes, lighters, mobile phones, and fire; the targets including people include at least people, people with cigarettes in their mouths, people holding cigarettes, people holding lighters, people holding mobile phones, people running, and people with abnormal limbs; Step B: monitoring events; The steps to monitor an event are as follows: Step B.1, identifying the target object based on the neural network to obtain an identification set; Step B.2, monitoring events according to the identification set; events refer to smoke, fire, and people with abnormal behavior; abnormal behavior includes holding a cigarette in the mouth, holding a cigarette, holding a lighter, holding a mobile phone, running, and abnormal limb state; Step B.3: If an event is detected, go to step B.4; otherwise go to step B.1; Step B.4: Event processing; after completing this step, go to step B.

1.

2. The visual-based gas station safety monitoring method according to claim 1 is characterized in that: The steps of identifying the target object based on the neural network and obtaining the identification set are as follows: The intelligent analyzer captures a frame of image Img from the video image output by the camera; wherein the width of the image Img is W and the height is H, and W and H need to satisfy formula (1), W ≥ Dw × RegPnX / MinW H≥Dh×RegPnY / MinH (1) (1) Wherein, Dw and Dh are the width and height of the camera monitoring area, respectively; RegPnX and RegPnY are the minimum pixel points in the X direction and the minimum pixel points in the Y direction of the object recognized by the neural network, respectively; MinW and MinH are the width and height of the minimum object that the neural network needs to recognize from the image Img, and MinW and MinH correspond to the actual width in the X direction and the actual height in the Y direction monitored by the camera, respectively; Compress the image Img into a compressed image; the compressed image is an image with a width of w and a height of h; w and h are the width and height of the image processed by the neural network, respectively, and w and h must satisfy formula (2), w≥MinW×W / ThingW h≥MinH×H / ThingH (2) (2) In the formula, ThingW and ThingH are the width and height of the smallest object that the neural network needs to recognize from the compressed image, and ThingW and ThingH correspond to the actual width in the X direction and the actual height in the Y direction monitored by the camera, respectively; Based on the neural network, the target objects are identified from the compressed image and the identification set is obtained; the elements in the identification set at least include the number of background target objects identified from the compressed image, the minimum circumscribed rectangle that frames each target object, and the identifier of each target object.

3. The vision-based gas station safety monitoring method according to claim 1 is characterized in that: The steps of monitoring events according to the identification set are as follows: According to the identification set, an event with smoke is monitored; the event with smoke is monitored when the number of background targets in the identification set is less than the maximum number of background targets; the maximum number of background targets is the number of background targets in the identification set when there is no smoke; According to the identification set, monitoring the fire event; the fire event is monitored when at least one target object identifier in the identification set is an identifier of the target object being fire; According to the identification set, events of people with abnormal behavior are monitored; events of people with abnormal behavior are monitored when at least one target object identifier in the identification symbol set is an identifier of a person with abnormal behavior, or the ASM condition must be met; the identification symbol set is a set of identifiers of target objects including people obtained from the identification set; the ASM condition means that if the abnormal behavior is running or abnormal limb state, the number of times the abnormal behavior is monitored within the set time SetTime must be ≥ NumK; wherein, 2s≤SetTime≤5s; NumK=SetTime×2times / s.

4. According to the vision-based gas station safety monitoring method of claim 1, the event processing includes generating event records, uploading event records, and saving event records, characterized in that: Event processing also includes safety prompts and event alarms.

5. The vision-based gas station safety monitoring method according to claim 1 is characterized in that: The steps of generating a neural network are as follows: Step 5.1, set the number of acquired pictures PNum, and set the processing set to an empty set; wherein PNum is a positive integer; Step 5.2, obtaining pictures; the pictures are pictures containing the target object and the number of obtained pictures is ≥ PNum; Step 5.3, obtaining a processing set using the image; Step 5.4, splitting the processing set into a training set and a test set; Step 5.5, training the neural network model using the training set; Step 5.6, using the test set to test the neural network model and counting the test data of each target object; the test data of the target object refers to the number of target object tests and the number of correct tests of the target object; Step 5.7, calculating the test accuracy of each target object according to the test data of each target object; Step 5.8: If the accuracy of each target test is ≥ the set accuracy, go to step 5.10; otherwise, go to step 5.9; Step 5.9, increase the PNum value, and then go to step 5.2; increasing the PNum value means that the PNum after increase = the PNum before increase × AddScale; wherein, 1.2≤AddScale≤2.0; Step 5.10, compiling the neural network model into a neural network; the neural network is a neural network that can recognize the target object; Step 5.11, write the neural network into each intelligent analyzer in the system.

6. The visual-based gas station safety monitoring method according to claim 5 is characterized in that: The steps of obtaining the picture are as follows: Acquire pictures from video recordings; wherein the video recordings are video recordings recorded by the network video recorder; and the pictures are pictures containing the target object and are different from each other; Get images containing the specified target object from the external network.

7. The vision-based gas station safety monitoring method according to claim 5, characterized in that: The steps of obtaining the processing set using the image are as follows: Annotation processing: Annotation processing refers to automatically annotating the unannotated pictures in the pictures using automatic annotation software to obtain the annotation file of the pictures; Correction processing: Correction processing refers to using modification annotation software to correct incorrect annotation files into correct annotation files; Compression processing; The compression process refers to compressing an uncompressed picture in the picture to obtain a compressed picture of the picture, and obtaining a labeling file of the compressed picture according to the labeling file of the picture; Using each set of pictures and annotation files, obtain a set target picture and an annotation file of the set target picture; the set of pictures and annotation files refers to a picture in the set and an annotation file of the picture; the picture in each set of pictures refers to a picture in the pictures; the set target picture is a picture that contains at least one set target and has a width of w and a height of h; A processing set is composed of each group of compressed images and annotation files and each group of set target images and annotation files.

8. The vision-based gas station safety monitoring method according to claim 7 is characterized in that: The steps of obtaining the set target object image and the set target object image's annotation file using each set of images and annotation files are as follows: Use each set of images and annotation files to obtain the set target object images; Using the set target object image, obtain a labeling file of the set target object image; The steps of obtaining the labeling file of the set target object image using the set target object image are as follows: Automatically annotate each unannotated set target object image using automatic annotation software to obtain an annotation file of the set target object image; Use the annotation modification software to correct the incorrectly set target object image annotation file into a correct annotation file.

Images