Catering safety monitoring system and method based on vision

Abstract

The invention discloses a catering safety monitoring system and method based on vision, and belongs to the technical field of visual catering safety monitoring. The method comprises the following steps: generating a neural network for identifying a target object; the neural network is used for monitoring the following events in the catering processing area: people with abnormal behaviors, fire, smoke, small animals, and incorrect placement of food material baskets, chopping boards and cutters; the abnormal behaviors comprise irregular dressing and tasting, smoking, using a mobile phone, running and abnormal limb states; when a person, a small animal, a chopping board and a cutter are monitored, an image with the width of w and the height of h is intercepted from the original image, and the incorrect position of the person, the small animal, the chopping board or the cutter with the abnormal behavior is recognized from the image through a neural network; the method has the beneficial effects that accurate and reliable monitoring can be implemented in the whole monitoring range by using the resolution of the original image, and the method can be widely applied to intelligent monitoring and supervision of catering processing places, and has relatively good economic benefits and wide market prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual monitoring of food safety, and in particular relates to a food safety monitoring system and method based on vision. Background Art

[0002] The Chinese invention patent number 202210673405.6 discloses a method and device for detecting the dress code of staff in an open kitchen scene. The training phase of the method includes the following steps: obtaining an input image through a camera, identifying the target pedestrian frame in the input image, obtaining pedestrian clothing information including hat wearing and apron wearing, and annotating the pedestrian clothing information to form a data set, and storing the annotated data samples, and dividing the data samples into training data and test data; an acquisition module, the acquisition module is used to encode the input image using the above method. The shortcomings of this method are: (1) there is no monitoring of the shoes and masks of kitchen staff; (2) there is no monitoring of the standardization of chefs' tasting behavior; (3) there is no monitoring of smoking and mobile phone use; (4) there is no monitoring of waste; (5) there is no monitoring of fire and smoke; (6) there is no monitoring of the standard placement of food baskets, cutting boards, and knives; (7) there is no monitoring of people not meeting the standards; (8) there is no monitoring of small animals such as mice, cats, weasels, and dogs; (9) monitoring is performed on compressed images, but after compression, smaller objects and details of objects in the original image have become smaller or disappeared, reducing the accuracy and reliability of monitoring; (10) there is no explanation of how to identify smaller objects or how to see the details of objects at a longer distance.

[0003] The Chinese invention patent number 202011550793.6 discloses a method for standard detection and identification of open kitchens and bright stoves based on artificial intelligence, including the following steps: S1: Collecting image data related to open kitchens and bright stoves; S2: Cleaning the images collected in step S1; S3: Using a model to predict and infer images of open kitchens and bright stoves; S4: Mixup the training images with the images that do not contain open kitchens and bright stoves in step S2; S5: Using the images of open kitchens and bright stoves to train the pp-yolo model; S6: Using the trained pp-yolo model to infer and predict the monitoring target objects; S6: Using the trained pp-yolo model to infer and predict the detection target objects, the target objects include chef clothes objects, chef hat objects, mask objects, mouse objects, and smoking objects, and using a personnel wearing fusion monitoring algorithm to determine whether each person wears the hat and clothes correctly. The shortcomings of this method are: (1) there is no monitoring of the shoes of kitchen staff; (2) there is no monitoring of the standardization of chefs' tasting behavior; (3) there is no monitoring of smoking and mobile phone use; (4) there is no monitoring of waste; (5) there is no monitoring of fire and smoke; (6) there is no monitoring of the standard placement of food baskets, cutting boards, and knives; (7) there is no monitoring of small animals such as cats, weasels, and dogs; (8) monitoring is performed on compressed images, but after compression, smaller objects and object details in the original image have become smaller or disappeared, reducing the accuracy and reliability of monitoring; (9) there is no monitoring of people who do not meet the standards; and (10) there is no explanation of how to identify smaller objects or at a longer distance and how to see the details of objects clearly. Summary of the invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a vision-based catering safety monitoring system and method, which is used to accurately identify various targets for monitoring catering safety in the entire catering processing area, and then implement reliable monitoring of catering safety.

[0005] The purpose of the present invention is achieved through the following technical solutions: The present invention provides a vision-based catering safety monitoring system, the system comprising a control server, N monitoring subsystems, T servers, V NVRs, a GPU server, a system switch, a network, a network device, a network device Wn, S mobile phone terminals, M clients and a system power supply; the control server is connected to the system switch, and is connected to the GPU server through the system switch; the control server is connected to the following devices through the system switch, the network and the network device: connected to each of the T servers, connected to each of the V NVRs, connected to the switch in each of the N monitoring subsystems and connected to the intelligent analyzer in the monitoring subsystem through the switch, the control server is connected to each of the S mobile phone terminals through the system switch and the network, connected to each of the M clients, connected to the network The device Wn is connected to the external network through the network device Wn; the server managing An(t) catering processing places among the T servers is connected to the switches in An(t) monitoring subsystems through the network device and the network, and is connected to the intelligent analyzer in the monitoring subsystem through the switch, t=1,2,...,T, and An(1)+An(2)+...+An(T)=N; the NVR connecting Bn(v) cameras among the V NVRs is connected to at least one switch in the monitoring subsystem through the network device and the network, and is connected to at least one camera in the monitoring subsystem through the switch, and a total of Bn(v) cameras are connected, v=1,2,...,V, and each Bn(v) is a positive integer; the system power supply is connected to the system switch and the network device Wn and provides the required DC power to the system switch and the network device Wn; wherein N, T, V, S and M are all positive integers.

[0006] The monitoring subsystem described in the present invention includes an intelligent analyzer, a switch, a network device, Q cameras and a power supply; wherein Q is a positive integer; the intelligent analyzer is connected to the switch, and is connected to the network device and each of the Q cameras through the switch, and is connected to the control server through the switch, the network device, the network, and the system switch, and is also connected to one of the T servers through the switch, the network device, and the network; the images collected by the Q cameras in the monitoring subsystem cover the entire area of ​​a catering processing; the power supply is connected to the intelligent analyzer, the switch, the network device, and provides the required DC power for the intelligent analyzer, the switch, and the network device.

[0007] The steps of the method described are as follows: Step A, generating a neural network; the neural network is a neural network that can identify targets; the targets include at least K targets; the K targets include 28 targets, M1 targets B1, M2 targets B2, and M2 targets B3; K, M1, and M2 are all positive integers; wherein, target 1, target 2, ..., target 28 among the 28 targets are respectively shelves, tables, cabinets, fixed barrels, stoves, range hoods, pillars, mice, cats, weasels, dogs, people, people in standard dress, people who taste food with special tasting utensils, people who wear hats improperly, people who wear masks improperly, people who wear tops improperly, people who wear pants improperly, people who wear shoes improperly, people who smoke cigarettes, people who use mobile phones, people who taste food with a cooking spoon, people who run, people with abnormal limbs, ground without waste, ground with waste The ground, fire, and smoke of the object; M1 target object B1 is the target object of the food basket of M1 color, M2 target object B2 is the target object of the cutting board of M2 color, and M2 target object B3 is the target object of the knife handle of M2 color; the people are not non-compliant people, and non-compliant people include people who wear hats irregularly, people who wear masks irregularly, people who wear tops irregularly, people who wear pants irregularly, people who wear shoes irregularly, people who smoke cigarettes, people who use mobile phones, people who taste food with cooking spoons, people who run, and people with abnormal limbs; the waste is waste with width ≥ ThingW and height ≥ ThingH; the fire is fire with width ≥ ThingW and height ≥ ThingH; the ThingW and ThingH correspond to the actual width in the X direction and the actual height in the Y direction of the image captured by the camera, respectively; Step B, monitoring and processing; The steps of monitoring and processing are as follows: Step B.1, set the initial value of q to 1; q is the number of the monitoring area in the monitoring subsystem; Step B.2: If the execution time of this step is within the monitoring time of the camera in the monitoring area q, go to step B.3; otherwise, go to step B.7; Step B.3, identify the target object based on the neural network and obtain the identification set IdenSet(q); Step B.4, monitoring events according to the identification set IdenSet(q) or monitoring events according to the difference graph; Step B.5: If an event is detected, go to step B.6; otherwise go to step B.7; Step B.6: event processing; Step B.7, adding 1 to the q; Step B.8: If q>Q, go to step B.1; otherwise go to step B.2; wherein Q is the number of monitoring areas in the monitoring subsystem.

[0008] The steps of identifying the target object based on the neural network and obtaining the identification set IdenSet(q) are as follows: Step 4.1, the intelligent analyzer captures a frame of image from the video image output by the camera of the monitoring area q; the width of the image is W(q) and the height is H(q); Step 4.2, compressing the image into a compressed image; the compressed image is an image with a width of w and a height of h; wherein w and h are the width and height of the image processed by the neural network, and w and h need to satisfy the following relationship: w≥DetectW×IdnXPn / ThingW h≥DetectH×IdnYPn / ThingH Wherein, DetectW and DetectH are the width and height of the camera monitoring area respectively; IdnXPn and IdnYPn are the minimum pixel point in the X direction and the minimum pixel point in the Y direction of the object recognized by the neural network respectively; Step 4.3, identifying the target object from the compressed image based on the neural network, and obtaining the identification set IdenSet(q); the elements in the identification set IdenSet(q) at least include the number of background targets identified from the compressed image, the minimum circumscribed rectangle that frames each target object, and the identifier of each target object; the identifier of target object k is the marker of target object k identified by the neural network, and the marker of target object k is a symbol assigned to target object k that is different from other targets, k=1,2,...,K; wherein k is the number of the target object; K is the number of the target objects; Step 4.4: End the process of identifying the target object based on the neural network and obtain the identification set IdenSet(q).

[0009] The steps of monitoring events according to the identification set IdenSet(q) or monitoring events according to the difference graph are as follows: According to the identification set IdenSet(q), monitoring smoke events; According to the identification set IdenSet(q), the steps of monitoring smoke events are as follows: According to the identification set IdenSet(q), the maximum number of background objects is recorded; the background objects include shelves, tables, cabinets, fixed barrels, stoves, range hoods and pillars; the number of background objects is the number of background objects in the identification set IdenSet(q); According to the identification set IdenSet(q) and the maximum number of background targets, an event of smoke is monitored; an event of smoke is detected when the number of background targets in the identification set IdenSet(q) is less than the maximum number of background targets; According to the identification set IdenSet(q), a fire event is monitored; the fire event is detected when at least one target object identifier in the identification set IdenSet(q) is an identifier of the target object being fire, and the target object is not within the range of any stove; According to the identification set IdenSet(q), an event of a ground with waste is monitored; the event of monitoring a ground with waste means that at least one identifier of a target object in the identification set IdenSet(q) is an identifier of a ground with waste; According to the identification set IdenSet(q), an event of incorrect food basket position is detected; the event of incorrect food basket position is detected when at least one target object identifier in the identification set IdenSet(q) is an identifier of the target object being the food basket, and the target object is not within the range box of the target object; the food basket is a food basket of M1 colors; If the execution time of this step is in the time period of monitoring small animals, then the event of monitoring the presence of small animals is detected according to the difference map; the event of detecting the presence of small animals means that at least one target object identifier in the small animal identification symbol set IdenAnimalSymSet(q) is an identifier of the target object being a small animal; the small animals include mice, cats, weasels, and dogs; According to the identification set IdenSet(q), an identification symbol set IdenPeopleSymSet(q) and a blade position identifier BdAndKnPlace(q) are obtained; wherein the elements in the identification symbol set IdenPeopleSymSet(q) are identifiers of target objects including people; According to the identification symbol set IdenPeopleSymSet(q), the event of a person who does not meet the specifications is monitored; the event of a person who does not meet the specifications is detected when at least one of the identifiers of the target object in the identification symbol set IdenPeopleSymSet(q) is an identifier of a person who does not meet the specifications, or the ASM condition is also satisfied; the target objects that do not meet the specifications include target 15, target 16, ..., target 24; the ASM condition is that if the identifier of the target object is an identifier of a continuous action target object, the event of a person who does not meet the specifications is detected when ActN / s≥SetMinN / s∧ ActTime≥MinTime; wherein, 2 times / s≤SetMinN / s≤5 times / s, 2s≤MinTime≤4s; ActN / s and SetMinN / s are respectively the number of times the identifier of the continuous motion target is detected within one second and the set minimum number of times the identifier of the continuous motion target is detected within one second; ActTime and MinTime are respectively the duration of the detected continuous motion target identifier and the set minimum duration of the detected continuous motion target identifier; the continuous motion targets are targets 23 and 24; According to the knife board position identifier BdAndKnPlace(q), the event of incorrect cutting board position or incorrect cutting tool position is monitored; the event of incorrect cutting board position or incorrect cutting tool position is detected when the knife board position identifier BdAndKnPlace(q)=false; the cutting board includes cutting boards of M2 colors; the cutting tools include cutting tools with handles of M2 colors.

[0010] The steps for setting the monitoring time of the camera in monitoring area q and the time period for monitoring small animals and the range frame of monitoring area q are as follows: For each camera in each monitoring area q, the monitoring start time and monitoring end time of each monitoring time period c in a day are set, and the time period for monitoring small animals is set, c=1,2,...,C(q), q=1,2,...,Q; where c is the number of the monitoring time period formed in the order of the monitoring time in a day; C(q) is the number of monitoring time periods in a day, and C(q) is a positive integer; The setting of the range frame of the monitoring area q refers to setting the range frame of the target object with a limited range in the camera acquisition image of each monitoring area q, q=1,2,...,Q; the range frame corresponds to the range of the actual stove, or corresponds to the correct range where the target object is actually placed; the targets with limited ranges include target 5 stove, target M1 color food basket, target M2 color chopping board, and target M2 color knife handle.

[0011] The steps of monitoring the event of small animals according to the difference map are as follows: Step 7.1, set the small animal identification symbol set IdenAnimalSymSet(q) to an empty set; where q is the number of the monitoring area; Step 7.2, obtaining a difference image from the video image; the difference image is a difference image obtained by subtracting the current frame image output by the camera of the monitoring area q from the previous L frames of image output; wherein 1≤L≤12; Step 7.3, obtaining a difference rectangle set DiffRectSet from the difference image; Step 7.4: If |DiffRectSet|≥1, go to step 7.5; otherwise, go to step 7.6; Step 7.5, identify and process the difference rectangle set DiffRectSet to obtain the small animal identification symbol set IdenAnimalSymSet(q); Step 7.6, end the processing of the event of monitoring the presence of small animals based on the difference map.

[0012] The steps of obtaining the identification symbol set IdenPeopleSymSet(q) and the blade position identifier BdAndKnPlace(q) according to the identification set IdenSet(q) are as follows: Step 8.1, set the initial value of n to 1, the initial value of n1 to 0, set the identification symbol set IdenPeopleSymSet(q) to an empty set, and set the blade position identifier BdAndKnPlace(q)=true; wherein n is the number of the target object in the identification set IdenSet(q); n1 is the number of the target object containing people in the identification set IdenSet(q); the targets containing people include target 12, target 13, ..., target 24; Step 8.2: If the identifier of the target object (n) is the identifier of a target object including a person, or the identifier of a target object including a chopping board, or the identifier of a target object including a knife, then go to step 8.3; otherwise, go to step 8.9; the target object including a chopping board includes targets of chopping boards of M2 colors; the target object including a knife includes targets of knives with handles of M2 colors; Step 8.3, obtaining an image ImgP(n) from the image according to the rectangle CompPRect(q,n); the image ImgP(n) is an image with a width of w and a height of h; the rectangle CompPRect(q,n) is the minimum circumscribed rectangle that frames the target object (n) in the identification set IdenSet(q); Step 8.4, using the neural network to identify the image ImgP(n), and obtaining the identifier IdenSym(q,n) of the image ImgP(n); Step 8.5: If the identifier of the target object (n) is an identifier of a target object including a person, go to step 8.6; otherwise, go to step 8.8; Step 8.6, add 1 to the n1; Step 8.7. Add the identifier IdenSym(q,n) to the identifier set with the name IdenPeopleSym(q,n1) IdenPeopleSymSet(q); After this step is completed, go to step 8.9; Step 8.8, obtain the knife plate position identifier BdAndKnPlace(q); The steps to obtain the blade position identifier BdAndKnPlace(q) are as follows: Step 8.8.1: If the target object corresponding to the identifier IdenSym(q,n) is not within the range box of the target object, go to step 8.8.2; otherwise, go to step 8.8.3; Step 8.8.2, set the knife position identifier BdAndKnPlace(q)=false; Step 8.8.3, end the process of obtaining the blade position identifier BdAndKnPlace(q); Step 8.9, add 1 to the n; Step 8.10: If n>Nc(q), go to step 8.11; otherwise go to step 8.2; wherein Nc(q) is the number of targets in the identification set IdenSet(q); Step 8.11, end the processing of obtaining the identification symbol set IdenPeopleSymSet(q) and the blade position identifier BdAndKnPlace(q) according to the identification set IdenSet(q).

[0013] The steps of obtaining the image ImgP(n) from the image according to the rectangle CompPRect(q,n) are as follows: Step 9.1, using formula (1) to calculate the left boundary value PSx(n), the right boundary value PEx(n), the upper boundary value PSy(n), and the lower boundary value PEy(n) of the rectangle CompPRect(q,n) in the rectangle PREct(n) of the image, PSx(n)=int[CPsx(n)×W(q) / w], PEx(n)=int[CPex(n)×W(q) / w] PSy(n)=int[CPsy(n)×H(q) / h], PEy(n)=int[CPey(n)×H(q) / h](1) (1) In the formula, CPsx(n), CPex(n), CPsy(n), and CPey(n) are the left boundary value, right boundary value, upper boundary value, and lower boundary value of the rectangle CompPRect(q,n), respectively; W(q) and H(q) are the width and height of the image, respectively; int[] is an integer operation; Step 9.2, use formula (2) to calculate the width Pw(n) and height Ph(n) of the rectangle PREct(n), Pw(n)=PEx(n)-PSx(n)+1 Ph(n)=PEy(n)-PSy(n)+1 (2) Step 9.3: If Pw(n)≤w∧Ph(n)≤h, go to step 9.4; otherwise go to step 9.5; Step 9.4, according to the boundary value of PREct(n), obtain the image ImgP(n) containing the entire rectangle PREct(n) from the image; after this step is completed, go to step 9.7; the image ImgP(n) is an image with a width of w and a height of h; Step 9.5, according to the boundary value of PREct(n), obtain the image EImgP(n) containing the entire rectangle PREct(n) from the image; the width of EImgP(n) is EPw(n) and the height is EpH(n); EPw(n) and EpH(n) are calculated by formula (3), EPw(n)=Pw(n),EPh(n)=int[h×Pw(n) / w] if Pw(n) / w≥Ph(n) / h EPw(n)=int[w×Ph(n) / h],EPh(n)=Ph(n) else (3) Step 9.6, compress the image EImgP(n) into an image ImgP(n) with a width of w and a height of h; Step 9.7, end the process of obtaining the image ImgP(n) from the image according to the rectangle CompPRect(q,n).

[0014] The steps of event processing are as follows: Generate event records; the generating of event records refers to the intelligent analyzer generating an event record set; the event record set includes at least the following elements: event name, time of event occurrence, location of event occurrence, event type, picture reflecting the event, or also includes GIF animation, or also includes short video, or also includes video recording reflecting the event; Uploading event records; the uploading event records refers to the intelligent analyzer uploading the event record set to the control server through the switch, network equipment, network, and system switch; Recording events: recording events means that the control server saves the event record set to the storage device of the control server; Event record publishing; event record publishing means that the control server publishes the event record set to the client with the permission to view the event record set and the mobile phone with the permission to view the event record set through the system switch and the network; Event alarm processing; the event alarm processing means that if the event is judged to be a smoke event, an alarm notification is made; if the event is judged to be a fire event, an alarm notification and event transmission are made; the alarm notification means that the control server sends the event information of smoke or fire to the relevant mobile phones and clients through the system switch and network; the event transmission means that the control server transmits the event record set to the system set place in the external network through the system switch, network, and network device Wn.

[0015] The steps of generating a neural network are as follows: Step 11.1, set the number of acquired pictures PicNum, and set the following sets as empty sets: atlas, preprocessing set, compression set, atlas of set target objects, processing set of set target objects, processing set; wherein PicNum is a positive integer ≥ 3000; the set targets include at least target 8, target 9, ..., target 26, targets of cutting boards of M2 colors, and targets of knives with knife handles of M2 colors; Step 11.2, obtain the atlas; Step 11.3, obtaining a processing set using the atlas; Step 11.4, splitting the processing set into a training set and a test set; Step 11.5, training the neural network model; training the neural network model means training the neural network model with each set of images and files in the training set; a set of images and files means a picture and a labeling file of the picture; Step 11.6, testing the neural network model; testing the neural network model means testing the neural network model with each set of images and files in the test set, and counting the number of each target object tested and the number of each target object tested correctly; Step 11.7, count the test accuracy of each target object k, k=1,2,...,K; Step 11.8: If the test accuracy of each target object k is ≥ the set accuracy, go to step 11.10; otherwise, go to step 11.9; Step 11.9, increase the PicNum value, and then go to step 11.2; increasing the PicNum value means that the PicNum after increase = PicNum before increase × ChgScaleV, where 1.1≤ChgScaleV≤2.4; Step 11.10, compiling the neural network model into a neural network; the neural network is a neural network that can identify the target object; Step 11.11, writing the neural network into each intelligent analyzer in the system; Step 11.12, end the process of generating the neural network.

[0016] The steps of obtaining the atlas are as follows: Obtaining video; obtaining video means obtaining D1 video segments containing the target object; D1 is a positive integer; Get pictures; Get pictures means to capture pictures from each video recording d1. d1=StrDx,StrDx+1,...,D1, and add the captured image to the temporary storage set TmpSet(d), d=StrD,StrD+1,...,D; where d1 is the number of the video recording segment without the captured image; StrDx is the starting number of the video recording segment without the captured image; d is the number of the temporary storage set; StrD is the starting number of the temporary storage set; D is the number of the temporary storage set, and D is a positive integer; Filter the images; filtering the images means filtering each temporary set TmpSet(d) that has not been filtered, d=StrD,StrD+1,...,D; filtering the temporary set TmpSet(d) means saving the different images in the temporary set TmpSet(d) into the atlas; two images in the temporary set TmpSet(d) are different if at least one of the targets in the two images is different, or the number of targets in the two images is different, or there is at least one target k such that PArea(d,a1,k)∩PArea(d,a2,k) is an empty set, a1≠a2∧k∈Z[1,K ]; wherein PArea(d,a1,k) and PArea(d,a2,k) are respectively the area where the target object k in the image Pt(d,a1) is located and the area where the same target object k in the image Pt(d,a2) is located, and the operation of PArea(d,a1,k)∩PArea(d,a2,k) is an operation performed on the image Pt(d,a1) and the image Pt(d,a2) in the same image coordinate system; wherein the image Pt(d,a1) and the image Pt(d,a2) are both images in the temporary set TmpSet(d); Z[m1,m2] is a set from integers m1 to integers m2, m1≤m2.

[0017] The steps of obtaining the processing set using the atlas are as follows: Annotation processing; Annotation processing refers to automatically annotating each unannotated picture in the atlas using automatic annotation software to obtain an annotation file of the picture, and then adding the picture and the annotation file of the picture to the preprocessing set; the annotation file at least includes the following data of each target object identified from the annotated picture: a rectangle marking the position of the target object, and a marker of the target object; wherein the marker of the target object is a different symbol assigned to different targets; Correction processing; Correction processing refers to correcting the uncorrected annotation files in the preprocessing set by using modification annotation software, and saving the corrected annotation files to the original annotation files in the preprocessing set; Correcting the annotation files means that if there are incorrect annotations in the annotation files, the annotation files are corrected to be correct annotation files; the correct annotation files at least include the following data of each target object in the annotated image: the minimum circumscribed rectangle that frames the target object, and the correct marker of the target object; Compression processing; compression processing refers to compressing each uncompressed image in the preprocessing set to obtain a compressed image of the image, and compressing the annotation file of the image to obtain an annotation file of the compressed image; then adding the compressed image and the annotation file of the compressed image to the compression set; the compressed image is an image with a width of w and a height of h; the annotation file of the compressed image is an annotation file that annotates the compressed image; Acquire a processing set of a set target object from the preprocessing set; The processing set is composed of the compression set and the processing set of the set target object.

[0018] The steps of obtaining the processing set of the set target object from the preprocessing set are as follows: Step 14.1, obtaining an atlas of a set target object from the preprocessing set; Step 14.2, using the atlas of the set target object to form a processing set of the set target object; The steps to construct a processing set of a set target object using a set of set target object images are as follows: Step 14.2.1, annotating the atlas of the set target object; annotating the atlas of the set target object means automatically annotating each unannotated picture in the atlas of the set target object using automatic annotation software, obtaining an annotation file of the picture, and adding the picture and the annotation file of the picture to the processing set of the set target object; Step 14.2.2, correct the processing set of the set target object; correcting the processing set of the set target object means using the modification annotation software to correct each uncorrected annotation file in the processing set of the set target object, and saving the correctly corrected annotation file to the original annotation file of the processing set of the set target object.

[0019] The steps of obtaining the atlas of the set target object from the preprocessing set are as follows: Step 15.1, set the initial value of pj to PStrJ, set the initial value of v to StrV, set PreProcPn to the number of images in the preprocessing set; pj is the number of the image that has not been taken in the preprocessing set; PStrJ is the starting number of the image that has not been taken in the preprocessing set; v is the number of the image in the set of the target object; StrV is the starting number of the image in the set of the target object; the image that has not been taken refers to the image of the set target object that has not been obtained; Step 15.2, obtaining a picture of a set target object from the picture P(pj) and the annotation file LabPFile(pj); the picture P(pj) and the annotation file LabPFile(pj) are both pictures and annotation files in the preprocessing set; The steps to obtain the image of the set target object from the image P(pj) and the annotation file LabPFile(pj) are as follows: Step 15.2.1, set the initial value of m to 1; wherein m is the number of the target object in the annotation file LabPFile(pj) in the preprocessing set; Step 15.2.2: If the identifier of the target object (m) is the identifier of the set target object, go to step 15.2.3; otherwise, go to step 15.2.7; Step 15.2.3, add 1 to the v; Step 15.2.4, extracting the minimum circumscribed rectangle Rect(m) that frames the target object (m) from the annotation file LabPFile(pj); Step 15.2.5, according to the minimum circumscribed rectangle Rect(m), obtain the picture APImg(v) from the picture P(pj); wherein the picture APImg(v) is a picture that contains the entire rectangle Rect(m) and has a width of w and a height of h; According to the minimum enclosing rectangle Rect(m), the steps of obtaining the image APImg(v) from the image P(pj) are as follows: Step 15.2.5.1, use formula (4) to calculate the width Rw(m) and height Rh(w) of the minimum circumscribed rectangle Rect(m), Rw(m)=REx(m)-RSx(m)+1 Rh(m)=REy(m)-RSy(m)+1 (4) (4) In the formula, RSx(m), REx(m), RSy(m), and REy(m) are the left boundary value, right boundary value, upper boundary value, and lower boundary value of the rectangle Rect(m), respectively; Step 15.2.5.2: If Rw(m)≤w∧Rh(m)≤h, go to step 15.2.5.3; otherwise go to step 15.2.5.4; Step 15.2.5.3, according to the boundary value of Rect(m), obtain the w*h area containing the entire rectangle Rect(m) from the image P(pj), and save the image of this w*h area as the image APImg(v); after this step is completed, go to step 15.2.5.5; wherein, the w*h area is an area with a width of w and a height of h; Step 15.2.5.4, according to the boundary value of Rect(m), obtain the ERw*ERh area containing the entire rectangle Rect(m) from the picture P(pj), compress the image of the ERw*ERh area into a w*h image, and then save the w*h image as the picture APImg(v); wherein the ERw*ERh area is an area with a width of ERw and a height of ERh; ERw and ERh are calculated by formula (5), ERw=Rw(m), ERh=int[h×Rw(m) / w] if Rw(m) / w≥Rh(m) / h ERw=int[w×Rh(m) / h], ERh=Rh(m) else (5) Step 15.2.5.5, end the process of obtaining the image API mg(v); Step 15.2.6, adding the image APImg(v) to the atlas of the set target object; Step 15.2.7, add 1 to the m; Step 15.2.8, if m>Mp(pj), go to step 15.2.9, otherwise go to step 15.2.2; where Mp(pj) is the number of targets in the annotation file LabPFile(pj); Step 15.2.9, end the process of obtaining the image of the set target object from the image P(pj) and the annotation file LabPFile(pj); Step 15.3, add 1 to the pj; Step 15.4: If pj>PreProcPn, go to step 15.5; otherwise go to step 15.2; Step 15.5, end the process of acquiring the atlas of the set target object.

[0020] Compared with the prior art, the significant progress and technical features of the vision-based catering safety monitoring system and method of the patent of the present invention are: the technical route of the present invention is, A, generate a neural network for identifying the target object, B, identify the target object based on the neural network to obtain an identification set, and then monitor events according to the identification set; the events include smoke, fire, waste on the ground, incorrect placement of food baskets, incorrect placement of cutting boards, incorrect placement of knives, the appearance of small animals, and people who do not meet the standards; when people, small animals, cutting boards, and knives are monitored, an image containing people, small animals, cutting boards, and knives with a width of w and a height of h is intercepted from the original image, and then the Yolov8 neural network identifies small animals, people who do not meet the standards, and incorrect cutting board positions or knife positions from the image; the non-compliance with standards includes irregular dress, irregular tasting of dishes, smoking, using mobile phones, running, and abnormal limb states, and during monitoring, small animals, human limbs, the color of the side of the cutting board, and the color of the knife handle are identified at the resolution of the original image; the technical purpose achieved is to greatly improve the content of monitoring and the accuracy and reliability of monitoring; the technical effect achieved is to provide fully intelligent and all-round monitoring for food hygiene and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the structure of a catering safety monitoring system according to an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of a monitoring subsystem according to an embodiment of the present invention; Figure 3 A schematic diagram of a process of monitoring events according to an identification set according to an embodiment of the present invention; Figure 4 A schematic diagram of a process of obtaining an identification symbol set of a target object including a person and obtaining a blade position identifier according to an identification set according to an embodiment of the present invention; Figure 5 A schematic diagram of a neural network generation process according to an embodiment of the present invention; Figure 6 This is an example image of obtaining a small animal from a difference image according to an embodiment of the present invention; Figure 7 This is an example of obtaining a w*h image containing a person from an original image according to an embodiment of the present invention. Figure 1 ; Figure 8 This is an example of obtaining a w*h image containing a person from an original image according to an embodiment of the present invention. Figure 2 ; Fig. 9 This is an example diagram of obtaining a w*h diagram including a cutting board and a knife from an original image according to an embodiment of the present invention.

[0022] The names of the numbers in the accompanying drawings are as follows: 1. isolated effective point 1, 2. isolated effective point 2, 3. chain code 1, 4. chain code 2; 5. person using a mobile phone; 6. person running; 7. knife; 8. chopping board.

[0023] Figure 6 In the figure, both the isolated valid point 1 and the isolated valid point 2 have no valid points in their 8-neighborhoods; the minimum bounding rectangles of chain code 1 and chain code 2 are TmpRect(1) and TmpRect(2), respectively, TmpRect(1) is the valid rectangle DiffR(1); the width, height and center point of region DiffR(1) are DRw(1), DRh(1) and DRC(1), respectively; DiffPic(1) is an image containing the valid rectangle DiffR(1) with a width of w and a height of h.

[0024] Figure 7 In the figure, the area of ​​person 5 using the mobile phone is PREct(6), the width, height and center point of PREct(6) are Pw(6), Ph(6) and PREctC(6) respectively; the image containing the area PREct(6) is ImgP(6), and Pw(6)<w, Ph(6)<h.

[0025] Figure 8 In the figure, the area of ​​the running person 6 is PREct(8), the width and height of PREct(8) and the center point are PRw(8) and Ph(8) and PREctC(8) respectively; the extended area including the area PREct(8) is EImgP(8), the width and height of EImgP(8) are EPw(8) and EpH(8) respectively, and Ph(8)>h.

[0026] Fig. 9 In the figure, the area of ​​the tool 7 is PREct(4), and the center point of PREct(4) is PREctC(4); the image containing the area PREct(4) is ImgP(4); the area of ​​the cutting board 8 is PREct(5), and the width, height and center point of PREct(5) are Pw(5), Ph(5) and PREctC(5) respectively; the image containing the area PREct(5) is ImgP(5). DETAILED DESCRIPTION

[0027] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to make the present invention more comprehensive and complete and to fully convey the concepts of example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and their repeated description will be omitted.

[0028] The words expressing positions and directions described in the present invention are all explained with reference to the accompanying drawings as examples, but they can be changed as needed, and all such changes are included in the protection scope of the present invention.

[0029] Before introducing the embodiment scheme, the parameters and terms used in the embodiment are first introduced: Set: A set is a set composed of some elements. The number of elements in the set Set is represented by |Set|. b∈Set means that b belongs to the set Set, or that b is an element in the set Set, and ∈ means belongs to; the empty set Set means that there are no elements in the set Set, and |Set|=0.

[0030] NVR: NVR is a network video recorder used to record and forward video footage from network cameras connected to it.

[0031] Pixel point: Pixel point p(x,y) is a pixel point in the image area PicWH; the value range of the X direction of the image area PicWH is 0 to W-1, and the value range of the Y direction is 0 to H-1; p(x,y)∈PicWH means that the pixel point p(x,y) takes every pixel in the image area PicWH.

[0032] Image point: an image point is an image point with an image value G(x,y) on a pixel point p(x,y); for example, a grayscale image point is an image point with a grayscale value Gray(x,y) on a pixel point p(x,y), 0≤Gray(x,y)≤V1; a color image point is an image point with a color value Color(x,y) formed by an image point with a red value Red(x,y), an image point with a green value Green(x,y) and an image point with a blue value Blue(x,y) on a pixel point p(x,y), 0≤Red(x,y)≤V2, 0≤Green(x,y)≤V2, 0≤Blue(x,y)≤V2; wherein, V1≥255, V2≥255, in this embodiment, V1=255, V2=255.

[0033] Image: An image is an image composed of each image point G(x,y). p(x,y)∈PicWH; For example, a grayscale image is a grayscale image composed of image points with grayscale values ​​Gray(x,y). p(x,y)∈PicWH; a color image is a color image consisting of image points with color values ​​Color(x,y). p(x,y)∈PicWH.

[0034] Image coordinate system: The image coordinate system is a coordinate system with the upper left corner of the image as the origin O of the image coordinate system, the upper edge of the image as the X-axis and the positive X-axis pointing from point O to the right, and the left edge of the image as the Y-axis and the positive Y-axis pointing from point O to the bottom.

[0035] Chain code algorithm: an algorithm that describes a curve using the coordinates of the curve's starting point and the boundary point direction code; Video image: Video image refers to an image that is output continuously at a certain rate (such as 25 frames / s).

[0036] Embodiment 1 to Embodiment 13, a vision-based catering safety monitoring system, the system includes a control server, N monitoring subsystems, T servers, V NVRs, a GPU server, a system switch, a network, a network device, a network device Wn, S mobile terminals, M clients and a system power supply; the control server is connected to the system switch, and connected to the GPU server through the system switch; the control server is connected to the following devices through the system switch and the network and the network device: connected to each of the T servers, connected to each of the V NVRs, connected to the switch in each of the N monitoring subsystems and connected to the intelligent analyzer in the monitoring subsystem through the switch, the control server is connected to each of the S mobile terminals through the system switch and the network, connected to each of the M clients, connected to the network device Wn and connected through the network The device Wn is connected to the external network; the server managing An(t) catering processing places among the T servers is connected to the switches in An(t) monitoring subsystems through the network device and the network, and is connected to the intelligent analyzer in the monitoring subsystem through the switch, t=1,2,...,T, and An(1)+An(2)+...+An(T)=N; the NVR connecting Bn(v) cameras among the V NVRs is connected to at least one switch in the monitoring subsystem through the network device and the network, and is connected to at least one camera in the monitoring subsystem through the switch, and a total of Bn(v) cameras are connected, v=1,2,...,V, and each Bn(v) is a positive integer; the system power supply is connected to the system switch and the network device Wn and provides the required DC power supply to the system switch and the network device Wn; wherein N, T, V, S and M are all positive integers; in this embodiment, N=80, T=9, V=10, S=32, M=15; The control server is a server that meets the use requirements and has a hard disk, an input device, and a display. The hard disk of the control server is installed with a Windows operating system, a database MySQL, and a first application software; the first application software at least includes the following software and programs: database management software, video download software, screenshot software, picture management software, picture filtering program, automatic marking software, marking modification software, splitting program, monitoring time and monitoring content and monitoring area range box setting program, network communication program; The server is a server for managing and processing information and data of the monitoring subsystem connected thereto; the server is a server having a hard disk, an input device, and a display to meet the use requirements, and the Windows operating system and the second application software are installed on the hard disk; the second application software at least includes the following software and programs: video downloading software, screenshot software, picture management software, picture filtering program, automatic marking software, marking modification software, splitting program, monitoring time and monitoring content and monitoring area range frame setting program, network communication program; The GPU server is a GPU server that meets the use requirements and has a hard disk on which a Linux operating system and a third application software are installed, and has an input device and a display, and is used for training neural network models, testing neural network models, compiling neural network models, deep learning, and video encoding and decoding. The third application software includes at least the following software and programs: training neural network model program, testing neural network model program, compilation software, and network communication program; Each of the V NVRs records the video footage of the camera connected thereto according to a set time period; The network device is one of the following devices or a combination of devices: a switch, a fiber optic transceiver, a switch and a fiber optic transceiver, a wireless network device, a switch and a wireless network device, and the wireless network device includes a wireless router, a wireless bridge, and a wireless wifi device; the system power supply is connected to the system switch and the network device Wn and provides the required DC power for the system switch and the network device Wn.

[0037] In Example 1 to Example 13, the monitoring subsystem includes an intelligent analyzer, a switch, a network device, Q cameras and a power supply; wherein Q is a positive integer; the intelligent analyzer is connected to the switch, and is connected to the network device and each of the Q cameras through the switch, and is connected to the control server through the switch, the network device, the network, and the system switch, and is also connected to one of the T servers through the switch, the network device, and the network; the images collected by the Q cameras in the monitoring subsystem cover the entire area of ​​a catering processing; the power supply is connected to the intelligent analyzer, the switch, and the network device, and provides the required DC power for the intelligent analyzer, the switch, and the network device; the specific value of Q depends on the size of the monitored catering processing area, usually 2≤Q≤5; The intelligent analyzer is an intelligent analyzer that meets the use requirements and has a hard disk and a Linux operating system and a fourth application software installed on the hard disk. The fourth application software at least includes the following algorithms and programs: download video software, image processing program, Yolov8 neural network program, picture shooting program, GIF animation program, short video program, video recording program, network communication program; the image processing program at least includes an image capture program, a color image conversion program to a grayscale image program, an image compression program, an image expansion program, a difference image acquisition program, a filtering algorithm program, a gradient algorithm program, an edge point acquisition program, a chain code algorithm program, an image area acquisition program, an image area intersection acquisition program, and a target object minimum circumscribed rectangle acquisition program; the Yolov8 neural network is a neural network that can at least identify each of the target objects; Each of the Q cameras is a network camera, or is converted into a camera that can connect to the network using a network conversion device; each of the Q cameras is a camera that works with 220V AC power, or is a camera that uses a power converter that comes with the camera to convert 220V AC power into the DC power required by the camera.

[0038] In embodiments 1 to 13, the present invention provides a method for monitoring food safety based on vision, which is implemented by the food safety monitoring system based on vision of the present invention, and the steps of the method are as follows: A. Generate neural network; Embodiment 1 and embodiment 2 generate a neural network; the neural network is a neural network that can identify targets; the targets include at least 48 targets; the 48 targets include 28 targets, 6 targets B1, 7 targets B2, and 7 targets B3; (K=48, M1=6, M2=7); wherein, target 1, target 2, ..., target 48 among the 48 targets are respectively shelves, tables, cabinets, fixed barrels, stoves, range hoods, pillars, mice, cats, weasels, dogs, people, people dressed in standard clothes, people who taste food with special tasting utensils, people who wear hats improperly, people who wear masks improperly, people who wear tops improperly, people who wear pants improperly, people who wear shoes improperly, people who smoke cigarettes, people who use mobile phones, people who taste food with cooking spoons , people who are running, people with abnormal limbs, ground without waste, ground with waste, fire, smoke; red food basket, blue food basket, green food basket, white food basket, yellow food basket, black food basket, red chopping board, green chopping board, white chopping board, blue chopping board, yellow chopping board, brown chopping board, black chopping board; knives with red handles, knives with green handles, knives with white handles, knives with blue handles, knives with yellow handles, knives with brown handles, knives with black handles; the people described are not people who do not meet the standards, and people who do not meet the standards include people who wear hats that are not standard, people who wear masks that are not standard, people who wear tops that are not standard, people who wear pants that are not standard, people who wear shoes that are not standard, people who smoke, people who use mobile phones, people who taste food with a cooking spoon, people who run, and people with abnormal limbs; The people who dress in accordance with the standard refer to those who wear hats in accordance with the standard, masks in accordance with the standard, tops in accordance with the standard, pants in accordance with the standard and shoes in accordance with the standard; the hat-wearing standard refers to the hat being worn on the head; the mask-wearing standard refers to the mask being worn on the mouth and covering at least the lower half of the nose; the top-wearing standard, pants-wearing standard and shoes-wearing standard refer to wearing tops, pants and shoes of the same style respectively; Ingredient baskets of different colors are used to store different ingredients; Different colored chopping boards refer to chopping boards with different colors on the sides. Different colored chopping boards are used to place different ingredients; knives with different colored handles are used to cut different ingredients; The person with abnormal limbs refers to a person whose limbs have at least one of the following conditions: the hand is clenched into a fist, the wrist is higher than the elbow, the ankle is straight, the ankle is higher than the knee, the angle between the forearm and the upper arm is less than 45°, and the angle between the thigh and the upper body is less than 135°; A runner is a person who runs with both feet off the ground at the same time; The waste is waste with a width ≥ ThingW and a height ≥ ThingH; the fire is fire with a width ≥ ThingW and a height ≥ ThingH; in this embodiment, ThingW=83mm, ThingH=83mm; In this embodiment, the completion time of generating the neural network is 2024 / 05 / 08 / 15:20; B. Monitoring and processing; Example 3 to Example 12, monitoring process; the start time of the monitoring process is 2024 / 05 / 09 / 08:00:00; The steps of monitoring and processing are as follows: Step B.1, set the initial value of q to 1; q is the number of the monitoring area in the monitoring subsystem; In this embodiment, the initial value of q is set to 1; Step B.2: If the execution time of this step is within the monitoring time of the camera in the monitoring area q, go to step B.3; otherwise, go to step B.7; In embodiments 3 to 11, the execution time of this step is within the monitoring time of the camera in the monitoring area q, and the process goes to step B.3; Example 12, when q=1, the execution time of this step is 09:00, which is not within the monitoring time of the camera in monitoring area 1, and the process goes to step B.7; Step B.3, identify the target object based on the neural network and obtain the identification set IdenSet(q); In Example 3 to Example 11, the target object is identified based on the neural network to obtain the identification set IdenSet(q); Step B.4, monitoring events according to the identification set IdenSet(q) or monitoring events according to the difference graph; Embodiments 3 to 11 monitor events according to the identification set IdenSet(q) or monitor events according to the difference graph; Step B.5: If an event is detected, go to step B.6; otherwise go to step B.7; In Examples 3 to 9 and 11, an event is detected, and the process goes to step B.6; Example 10: No event is detected, go to step B.7; Step B.6: event processing; Embodiments 3 to 9 and 11, event processing; Step B.7, adding 1 to the q; Step B.8: If q>Q, go to step B.1; otherwise go to step B.2; wherein Q is the number of monitoring areas in the monitoring subsystem.

[0039] In Example 3 to Example 11, the steps of identifying the target object based on the neural network and obtaining the identification set IdenSet(q) are as follows: Step 4.1, the intelligent analyzer captures a frame of image from the video image output by the camera of the monitoring area q; the width of the image is W(q) and the height is H(q); In this embodiment, the intelligent analyzer captures a frame of image Pic(q) from the video image output by the camera; when q=1, the width of the image Pic(1) is W(1)=1920 and the height is H(1)=1080; Step 4.2, compress the image Pic(q) into a compressed image CompP(q); the compressed image is an image with a width of w and a height of h; wherein w and h are the width and height of the image processed by the neural network, and w and h need to satisfy the following relationship: w≥DetectW×IdnXPn / ThingW h≥DetectH×IdnYPn / ThingH Wherein, DetectW and DetectH are the width and height of the camera monitoring area respectively; IdnXPn and IdnYPn are the minimum pixel point in the X direction and the minimum pixel point in the Y direction of the object recognized by the neural network respectively; In this embodiment, the image Pic(1) is compressed into a compressed image CompP(1); the compressed image CompP(1) is an image with a width of w and a height of h, and w and h need to satisfy the following relationship: w≥DetectW×IdnXPn / ThingW=10000mm×5 / 83mm=602.41 h≥DetectH×IdnYPn / ThingH=10000mm×5 / 83mm=602.41 Wherein, DetectW=10m, DetectH=10m; IdnXPn=IdnYPn=5; ThingW=ThingH=83mm; In this embodiment, the width w=608>602.41 and the height h=608>602.41 of the image processed by the Yolov8 neural network; It can be seen that this system can detect fire, waste and other objects with a width of ≥83mm and a height of ≥83mm within a range of 10m in width and 10m in height using the Yolov8 neural network; the size of this identified object is sufficient to meet the needs of actual monitoring; Step 4.3, identifying the target object from the compressed image based on the neural network, and obtaining the identification set IdenSet(q); the elements in the identification set IdenSet(q) at least include the number of background targets identified from the compressed image, the minimum circumscribed rectangle that frames each target object, and the identifier of each target object; the identifier of target object k is the marker of target object k identified by the neural network, and the marker of target object k is a symbol assigned to target object k that is different from other targets, k=1,2,...,K; wherein k is the number of the target object; K is the number of the target objects; In this embodiment, the target object is identified from the compressed image CompP(q) based on the Yolov8 neural network, and the identification set IdenSet(q) of the compressed image CompP(q) is obtained; IdenSet(q)={IdenBGsN(q),CompPRect(q,n),CompISym(q,n)|n=1,2,...,Nc(q)}; the identification set The elements in IdenSet(q) include the number of background objects identified from the compressed image IdenBGsN(q), the minimum bounding rectangle CompPRect(q,n) that frames each object (n), and the identifier CompISym(q,n) of each object (n), n=1,2,...,Nc(q); wherein n is the number of objects in the compressed image CompP(q) formed in increasing order of the center points of the minimum bounding rectangle CompPRect(q,n) of the object, Nc(q) is the number of objects in the compressed image CompP(q), and Nc(q) is a positive integer; The numbering formed in increasing order of the points refers to the numbering formed according to the y value of the points in the image coordinate system from small to large, and then according to the x value from small to large; the image coordinate system is a coordinate system with the upper left corner of the image as the origin O, the upper edge of the image as the X-axis and the positive X-axis pointing from the origin O to the left, the left edge of the image as the Y-axis and the positive Y-axis pointing from the origin O to the bottom; the identifier of the target object is the marker of the target object identified by the neural network; in this embodiment, the marker of the target object k is a symbol Symbol(k)=k given to the target object k, which is different from other targets, k=1,2,...,48; the image coordinate system refers to the image coordinate system of the compressed image CompP(q); Step 4.4: End the process of identifying the target object based on the neural network and obtain the identification set IdenSet(q).

[0040] In this embodiment, the target object is identified based on the neural network and the process of obtaining the identification set IdenSet(q) is completed.

[0041] In Example 3 to Example 11, the steps of monitoring events according to the identification set IdenSet(q) or monitoring events according to the difference graph are as follows: According to the identification set IdenSet(q), monitoring smoke events; Example 3, when q=2, according to the identification set IdenSet(2), monitoring the event of smoke; According to the identification set IdenSet(q), the steps of monitoring smoke events are as follows: According to the identification set IdenSet(q), the maximum number of background objects is recorded; the background objects include shelves, tables, cabinets, fixed barrels, stoves, range hoods and pillars; the number of background objects is the number of background objects in the identification set IdenSet(q); In this embodiment, when q=2, according to the identification set IdenSet(2), recording the maximum number of background targets means that when the time is 2024 / 05 / 09 / 08:00:00, this step is executed for the first time, and the maximum number of background targets MaxBn(2)=IdenBGsN(2)=6 is set; wherein IdenBGsN(2) is the number of background targets in the identification set IdenSet(2)=6; this step is executed after 2024 / 05 / 09 / 08:00:00, and MaxBn(2)=Max(MaxBn(2),IdenBGsN(2)); wherein IdenBGsN(2) is the number of background targets in the identification set IdenSet(2); Max() is a maximum value operation; According to the identification set IdenSet(q) and the maximum number of background targets, an event of smoke is monitored; an event of smoke is detected when the number of background targets in the identification set IdenSet(q) is less than the maximum number of background targets; Example 3, when q=2, the execution time of this step is 2024 / 06 / 05 / 12:35:07, according to the identification set IdenSet(2) and the maximum number of background targets MaxBn(2), a smoke event is detected; the smoke event means that the number of background targets in the identification set IdenSet(2) IdenBGsN(2)=5<the maximum number of background targets MaxBn(2)=6; because there is smoke, there is no target range hood in the identification set IdenSet(2), so the number of background targets in the identification set IdenSet(2) IdenBGsN(2)=5; the time when the smoke event occurs is 2024 / 06 / 05 / 12:35:07; According to the identification set IdenSet(q), a fire event is monitored; the fire event is detected when at least one target object identifier in the identification set IdenSet(q) is an identifier of the target object being fire, and the target object is not within the range of any stove; Example 4, when q=1, according to the identification set IdenSet(1), a fire event is detected; the fire event is detected when there is an identifier of a target object (1) in the identification set IdenSet(1) that is the identifier of the target object being fire CompISym(1,1)=27, and the target object (1) is not within the range frame LimitR(1,1) of the stove in the monitoring area 1; the time when the fire event occurs is 2024 / 06 / 07 / 08:32:09; According to the identification set IdenSet(q), an event of a ground with waste is monitored; the event of monitoring a ground with waste means that at least one identifier of a target object in the identification set IdenSet(q) is an identifier of a ground with waste; Example 5, when q=2, according to the identification set IdenSet(2), an event of detecting a ground with waste is detected; the event of detecting a ground with waste means that in the identification set IdenSet(2), there is an identifier of a target object (3) whose target object is the ground with waste, and the identifier CompISym(2,3)=26; the time when the event of detecting a ground with waste occurs is 2024 / 06 / 08 / 16:39:55; According to the identification set IdenSet(q), an event of incorrect food basket position is detected; the event of incorrect food basket position is detected when at least one target object identifier in the identification set IdenSet(q) is an identifier of the target object being the food basket, and the target object is not within the range box of the target object; the food basket is a food basket of M1 colors; Example 6, when q=1, according to the identification set IdenSet(1), an event of incorrect food basket position is detected; the event of incorrect food basket position is detected when there is an identifier of a target object (3) in the identification set IdenSet(1) that is the identifier of the target object being a blue food basket CompISym(1,3)=30, and the target object (3) is not within the range box LimitR(1,2) of the target object (3); the time when the event of incorrect food basket position is detected is 2024 / 05 / 22 / 07:26:12; If the execution time of this step is in the time period of monitoring small animals, then the event of monitoring the presence of small animals is detected according to the difference map; the event of detecting the presence of small animals means that at least one target object identifier in the small animal identification symbol set IdenAnimalSymSet(q) is an identifier of the target object being a small animal; the small animals include mice, cats, weasels, and dogs; Example 7, when q=1, the execution time of this step is 2024 / 05 / 20 / 23:15:29, which is in the time period for monitoring small animals (the starting monitoring time of the fifth monitoring time period is StrTime(1,5)=21:30, the ending monitoring time is EndTime(1,5)=23:59, and it is the time period for monitoring small animals DtAn(1,5)=1); this step detects the event of small animals, and the event of small animals is that the identifier of a target object (1) in the small animal identification symbol set IdenAnimalSymSet(1) is the identifier of the target object being a mouse, CompISym(1,1)=8; the time when the event of small animals is detected is 2024 / 05 / 20 / 23:15:29; According to the identification set IdenSet(q), an identification symbol set IdenPeopleSymSet(q) and a blade position identifier BdAndKnPlace(q) are obtained; wherein the elements in the identification symbol set IdenPeopleSymSet(q) are identifiers of target objects including people; Embodiment 8, when q=1, according to the identification set IdenSet(1), the identification symbol set IdenPeopleSymSet(1) and the blade position identifier BdAndKnPlace(1) are obtained; wherein BdAndKnPlace(1)=true; In Example 9 and Example 10, when q=2, the identification symbol set IdenPeopleSymSet(2) and the blade position identifier BdAndKnPlace(2) are obtained according to the identification set IdenSet(2); wherein BdAndKnPlace(2)=true; Embodiment 11, when q=2, according to the identification set IdenSet(2), the identification symbol set IdenPeopleSymSet(2) and the blade position identifier BdAndKnPlace(2)=false are obtained; wherein IdenPeopleSymSet(2) is an empty set; According to the identification symbol set IdenPeopleSymSet(q), the event of a person who does not meet the specifications is monitored; the event of a person who does not meet the specifications is detected when at least one of the identifiers of the target object in the identification symbol set IdenPeopleSymSet(q) is an identifier of a person who does not meet the specifications, or the ASM condition is also satisfied; the target objects that do not meet the specifications include target 15, target 16, ..., target 24; the ASM condition is that if the identifier of the target object is an identifier of a continuous action target object, the event of a person who does not meet the specifications is detected when ActN / s≥SetMinN / s∧ ActTime≥MinTime; wherein, 2 times / s≤SetMinN / s≤5 times / s, 2s≤MinTime≤4s; ActN / s and SetMinN / s are respectively the number of times the identifier of the continuous motion target is detected within one second and the set minimum number of times the identifier of the continuous motion target is detected within one second; ActTime and MinTime are respectively the duration of the detected continuous motion target identifier and the set minimum duration of the detected continuous motion target identifier; the continuous motion targets are targets 23 and 24; Example 8, when q=1, according to the identification symbol set IdenPeopleSymSet(1), an event of a person who does not meet the specifications is detected; the event of a person who does not meet the specifications is detected when the identification symbol set IdenPeopleSymSet(1) has an identifier of a target object (6) whose identifier is a person using a mobile phone, IdenPeopleSym(1,1)=21; the time when the event of a person who does not meet the specifications is detected is 2024 / 05 / 30 / 10:27:33; In Example 9 and Example 10, when q=2, according to the identification symbol set IdenPeopleSymSet(2), an event of a person who does not conform to the specification is detected; the event of a person who does not conform to the specification is detected when the identification symbol set IdenPeopleSymSet(2) has an identifier of a target object (8) which is an identifier of a person who does not conform to the specification IdenPeopleSym(2,1)=23, and also satisfies the ASM condition; the ASM condition is satisfied because the identifier of the target object (8) is an identifier of a continuous action target object 23 and satisfies ActN / s=2times / s≥SetMinN / s=2times / s∧ActTime=5s≥ The condition of MinTime=2s; the time when the event of detecting a person who does not meet the specification occurs is 2024 / 05 / 27 / 12:39:22; wherein, in the identification symbol set IdenPeopleSymSet(2), there is an identifier of a target object (9) which is an identifier of a person IdenPeopleSym(2,2)=12; According to the knife board position identifier BdAndKnPlace(q), the event of incorrect cutting board position or incorrect tool position is monitored; the event of incorrect cutting board position or incorrect tool position is detected when the knife board position identifier BdAndKnPlace(q)=false; the cutting board includes cutting boards of M2 colors; the cutting tools include cutting tools with handles of M2 colors; in this embodiment, M2=7.

[0042] Example 11, when q=2, according to the blade position identifier BdAndKnPlace(2), an event of incorrect tool position is detected; the event of incorrect tool position is detected when the blade position identifier BdAndKnPlace(2)=false; the reason is that the target object (4) is the target object 42 (red tool) and is not within the range box LimitR(2,3) of the target object (4); the time when the event of incorrect tool position is detected is: 2024 / 05 / 31 / 12:29:09.

[0043] Example 13, the steps of setting the monitoring time of the camera of monitoring area q and the time period of monitoring small animals and the range frame of monitoring area q are as follows: Step 6.1, set the initial value of q to 1; q is the number of the monitoring area; Step 6.2, set the monitoring start time StrTime(q,c) and the monitoring end time EndTime(q,c) of each monitoring time period c of the camera in the monitoring area q within a day, and set the time period DtAn(q,c) for monitoring small animals, c=1,2,...,C(q); wherein c is the number of the monitoring time period formed according to the order of the monitoring time within a day; C(q) is the number of monitoring time periods within a day, and C(q) is a positive integer; DtAn(q,c)=0 and DtAn(q,c)=1 correspond to not monitoring small animals in the monitoring time period c and monitoring small animals in the monitoring time period c, respectively; In this embodiment, when q=1, the monitoring start time StrTime(q,c) and the monitoring end time EndTime(q,c) of each monitoring time period c of the monitoring area 1 within one day are set, and the time period DtAn(q,c) for monitoring small animals is set, c=1,2,...,5; C(1)=5; the following are the specific data set, StrTime(1,1)=00:00, EndTime(1,1)=04:59, DtAn(1,1)=1; StrTime(1,2)=05:00, EndTime(1,2)=08:29, DtAn(1,2)=0; StrTime(1,3)=10:00, EndTime(1,3)=13:59, DtAn(1,3)=0; StrTime(1,4)=16:30, EndTime(1,4)=21:29, DtAn(1,4)=0; StrTime(1,5)=21:30, EndTime(1,5)=23:59, DtAn(1,5)=1; Step 6.3, setting the range frame of the monitoring area q; The steps for setting the scope box of monitoring area q are as follows: Step 6.3.1, set the initial value of f to 1; f is the number of the target objects with limited range in the monitoring area q according to the set rule; the target objects with limited range include target 5 stove, target objects of M1 color food basket, target objects of M2 color cutting board, target objects of M2 color knife handle; In this embodiment, the initial value of f is set to 1; f is a number formed in increasing order of the center points of the target objects with a limited range; the target objects with a limited range include target 5 stove, 6 color food basket targets, 7 color cutting board targets, and 7 color knife handle targets; Step 6.3.2, set the range frame LimitR(q,f) of the target object (f); the target object (f) is a target object with a limited range and a target number f; the range frame LimitR(q,f) corresponds to the actual correct placement range of the target object (f), or corresponds to the range of the actual stove; In this embodiment, when q=1 and f=1, a range frame LimitR(1,1) of the target object (1) is set, and the range frame LimitR(1,1) corresponds to the range of the actual target object 5, the stove; In this embodiment, when q=1 and f=2, a range frame LimitR(1,2) of the target object (2) is set, and the range frame LimitR(1,2) corresponds to the correct range where the blue food basket of the target object (2) is actually placed; In this embodiment, when q=2 and f=3, a range frame LimitR(2,3) of the target object (3) is set, and the range frame LimitR(2,3) corresponds to the correct range in which the tool with the red handle of the target object (3) is actually placed; Step 6.3.3, add 1 to the f; Step 6.3.4, if f>F(q), go to step 6.3.5; otherwise go to step 6.3.2; F(q) is the number of targets with a limited range in the monitoring area q; F(q) is a positive integer; in this embodiment, when q=1, F(1)=2; when q=2, F(2)=3; Step 6.3.5, ending the process of setting the range frame of the monitoring area q; Step 6.4, add 1 to the q; Step 6.5: If q>Q, go to step 6.6; otherwise go to step 6.2; Step 6.6: End the setting of the monitoring time of the camera in the monitoring area q and the time period for monitoring small animals and the processing of the range frame of the monitoring area q.

[0044] In Example 7, the steps of monitoring the event of small animals according to the difference map are as follows: Step 7.1, set the small animal identification symbol set IdenAnimalSymSet(q) to an empty set; where q is the number of the monitoring area; In this embodiment, when q=1, the small animal identification symbol set IdenAnimalSymSet(1) is set to an empty set; Step 7.2, obtaining a difference image from the video image; the difference image is a difference image obtained by subtracting the current frame image output by the camera of the monitoring area q from the previous L frames of image output; wherein 1≤L≤12; in this embodiment, L=1; The steps to obtain the difference image DiffImg from the video image are as follows: Grayscale the image ThisImg to obtain the grayscale image ThisGray of the image ThisImg; Grayscale the image LastImg of the previous frame to obtain the grayscale image LastGray of the image LastImg; The image value DiffGv(x,y) of each image point in the difference image DiffImg is calculated using the following formula: p(x,y)∈PicWH, DiffGv(x,y)=|ThisGray(x,y)-LastGray(x,y)| Where, || is the absolute value operation; ThisGray(x,y) and LastGray(x,y) are the image points in the grayscale image ThisGray and the image points in the grayscale image LastGray respectively; 0≤ThisGray(x,y)≤255, 0≤LastGray(x,y)≤255; p(x,y)∈PicWH means taking each pixel in the image area PicWH; PicWH is the image area of ​​the current frame and the image area of ​​the previous frame; Step 7.3, obtaining a difference rectangle set DiffRectSet from the difference image; The steps of obtaining the difference rectangle set DiffRectSet from the difference image are as follows: Step 7.3.1, set the initial value of Ev to 0, and set the difference rectangle set DiffRectSet to an empty set; Ev is the number of valid rectangles in the difference image; Step 7.3.2, obtaining a valid difference map ValidDiffImg from the difference map; wherein the image value ValidP(x, y) of each difference point in the valid difference map ValidDiffImg is calculated by the following formula: p(x,y)∈PicWH, ValidP(x,y)=DiffGv(x,y) if DiffGv(x,y)≥AveV ValidP(x,y)=0 else Wherein, the point where the image value ValidP(x, y)≠0 is a valid point; AveV=Max(MinV, Sv×AveDv); 5≤MinV≤12; MinV is the minimum valid difference value; AveDv is the average value of the image value DiffGv(x, y)≠0 among all the image values ​​in the difference image; 0.618≤Sv≤1.618; in this embodiment, MinV=7; Sv=1.050; Step 7.3.3, filtering the valid difference map ValidDiffImg; In this embodiment, the Gaussian filtering algorithm is used to filter the valid difference map ValidDiffImg; the Gaussian filtering algorithm is a filtering algorithm that deletes each isolated valid point in the valid difference map ValidDiffImg, and an isolated valid point means that there is no valid point in the 8-neighborhood of the point; Figure 6 As shown, the isolated valid point 1 and the isolated valid point 2 are deleted; Step 7.3.4: If there are valid points in the valid difference graph ValidDiffImg, go to step 7.3.5; otherwise, go to step 7.3.6; In this embodiment, there are valid points in the valid difference graph ValidDiffImg, and the process goes to step 7.3.5; Step 7.3.5, obtain a valid rectangle from the valid difference map ValidDiffImg; a valid rectangle is a rectangle that satisfies condition Vd; condition Vd includes that the width of the rectangle ≥ DiffMinW, the height of the rectangle ≥ DiffMinH, the distance between adjacent valid points in the rectangle ≤ DiffMinD; 2≤DiffMinW≤8, 2≤DiffMinH≤6, 2≤DiffMinD≤8; the units of DiffMinW, DiffMinH, and DiffMinD are all pixels; in this embodiment, DiffMinW=6, DiffMinH=4, and DiffMinD=3; The specific steps to obtain a valid rectangle from the valid difference map ValidDiffImg are as follows: Step 7.3.5.1, connect the valid points using the chain code algorithm; wherein the distance between two adjacent valid points is DiffMinD; obtain In chain codes i in the difference graph; i is the number of the chain code formed in the increasing order of the chain code starting point; In this embodiment, the chain code algorithm is used to connect the valid points to obtain chain code 1 and chain code 2 in the difference graph; In=2 (Appendix Figure 6 3, 4 in (); Step 7.3.5.2, according to the boundary value of each chain code i, obtain the minimum circumscribed rectangle TmpRect(i), i=1,2,...,In; In this embodiment, the minimum bounding rectangle TmpRect(1) of chain code 1 is obtained according to the boundary value of chain code 1, and the minimum bounding rectangle TmpRect(2) of chain code 2 is obtained according to the boundary value of chain code 2. Step 7.3.5.3, judge and process each minimum bounding rectangle TmpRect(i), i=1,2,...,In, and obtain Ev valid rectangles DiffR(e), e=1,2,...,Ev; wherein e is the number of the valid rectangles in increasing order of the center points of the valid rectangles; Ev is the number of valid rectangles; the valid rectangle is a rectangle with a width ≥ DiffMinW and a height ≥ DiffMinH; In this embodiment, a valid rectangle DiffR(1) is obtained; Ev=1 (because the width of TmpRect(2)=3<DiffMinW=6, so the minimum enclosing rectangle TmpRect(2) is not a valid rectangle); Step 7.3.5.4: If Ev≥1, go to step 7.3.5.5; otherwise go to step 7.3.5.6; In this embodiment, Ev=1≥1, go to step 7.3.5.5; Step 7.3.5.5, add the valid rectangles DiffR(1), DiffR(2), ..., DiffR(Ev) to the difference rectangle set DiffRectSet; In this embodiment, the valid rectangle DiffR(1) is added to the difference rectangle set DiffRectSet; Step 7.3.5.6, end the process of obtaining the valid rectangle from the valid difference map ValidDiffImg; In this embodiment, the process of obtaining a valid rectangle from the valid difference map ValidDiffImg is finished; wherein the valid rectangle DiffRectSet={DiffR(1)}; Step 7.3.6, ending the process of obtaining the difference rectangle set DiffRectSet from the difference image; Step 7.4: If |DiffRectSet|≥1, go to step 7.5; otherwise go to step 7.6; In this embodiment, |DiffRectSet|=1≥1, go to step 7.5; Step 7.5, identify and process the difference rectangle set DiffRectSet to obtain the small animal identification symbol set IdenAnimalSymSet(q); In this embodiment, when q=1, the steps of identifying and processing the difference rectangle set DiffRectSet to obtain the small animal identification symbol set IdenAnimalSymSet(1) are as follows: Step 7.5.1, set the initial value of e to 1; e is the number of the valid rectangle in the difference rectangle set DiffRectSet; In this embodiment, the initial value of e is set to 1; Step 7.5.2: If DRw(e)≤w∧DRh(e)≤h, go to step 7.5.3; otherwise go to step 7.5.4; where DRw(e) and DRh(e) are the width and height of the effective rectangle DiffR(e), respectively. In this embodiment, when e=1, DRw(1)≤w∧DRh(1)≤h, go to step 7.5.3; Step 7.5.3, according to the boundary value of the effective rectangle DiffR(e), obtain the image DiffPic(e) containing the entire effective rectangle DiffR(e) from the current frame image; the image DiffPic(e) is an image with a width of w and a height of h. After this step is completed, proceed to step 7.5.6; In this embodiment, according to the boundary value of the effective rectangle DiffR(1), an image DiffPic(1) including the entire effective rectangle DiffR(1) is obtained from the current frame image ThisImg with the center point DRC(1) of DiffR(1) as the center (see the attached figure). Figure 6 After this step is completed, proceed to step 7.5.6; Step 7.5.4, according to the boundary value of the effective rectangle DiffR(e), obtain the image EDiffPic(e) containing the entire effective rectangle DiffR(e) from the current frame image; the width of EDiffPic(e) is EDRw(e) and the height is EDRh(e), and EDRw(e) and EDRh(e) are calculated by formula (2), EDRw(e)=DRw(e), EDRh(e)=int[h×DRw(e) / w] if DRw(e) / w≥DRh(e) / h EDRw(e)=int[w×DRh(e) / h], EDRh(e)=DRh(e) else (2) Step 7.5.5, compress the image EDiffPic(e) into an image DiffPic(e) with a width of w and a height of h; Step 7.5.6, using the neural network to identify the image DiffPic(e), and obtaining the identifier CompISym(q,e) of the target object (e) contained in the image DiffImg(e); In this embodiment, when q=1, e=1, the neural network is used to identify the image DiffPic(1), and the identifier CompISym(1,1)=8 of the target object (1) mouse contained in the image DiffPic(1) is obtained; Step 7.5.7, adding the identifier CompISym(q,e) of the target object (e) to the small animal identification symbol set IdenAnimalSymSet(q); In this embodiment, when q=1, the identifier CompISym(1,1)=8 of the target object (1) is added to the small animal identification symbol set IdenAnimalSymSet(1); Step 7.5.8, add 1 to the e; In this embodiment, 1 is added to the e; e plus 1 is 2; Step 7.5.9: If e>|DiffRectSet|, go to step 7.5.10; otherwise go to step 7.5.2; In this embodiment, e=2>|DiffRectSet|=1, go to step 7.5.10; Step 7.5.10, ending the processing of the difference rectangle set DiffRectSet described in the identification processing; Step 7.6, end the processing of the event of monitoring the presence of small animals based on the difference map.

[0045] In this embodiment, after this step is completed, IdenAnimalSymSet(1)={CompISym(1,1)|e=1}; Ev=1, CompISym(1,1)=8.

[0046] In Example 8 to Example 10, the steps of obtaining the identification symbol set IdenPeopleSymSet(q) and the blade position identifier BdAndKnPlace(q) according to the identification set IdenSet(q) are as follows: Step 8.1, set the initial value of n to 1, the initial value of n1 to 0, set the identification symbol set IdenPeopleSymSet(q) to an empty set, and set the blade position identifier BdAndKnPlace(q)=true; wherein n is the number of the target object in the identification set IdenSet(q); n1 is the number of the target object containing people in the identification set IdenSet(q); the targets containing people include target 12, target 13, ..., target 24; Example 8: When q=1, set n=1, n1=0, set the identification symbol set IdenPeopleSymSet(1) to an empty set, and set BdAndKnPlace(1)=true; the execution time of this step is 2024 / 05 / 30 / 10:27:33; In Example 9 and Example 10, when q=2, n=1, n1=0, the identification symbol set IdenPeopleSymSet(2) is set to an empty set, and BdAndKnPlace(2)=true is set; the execution time of this step is 2024 / 05 / 27 / 12:39:22; Example 11, when q=2, set n=1, n1=0, set the identification symbol set IdenPeopleSymSet(2) to an empty set, and set BdAndKnPlace(2)=true; the execution time of this step is 2024 / 05 / 31 / 12:29:09; Step 8.2: If the identifier of the target object (n) is the identifier of a target object including a person, or the identifier of a target object including a chopping board, or the identifier of a target object including a knife, then go to step 8.3; otherwise, go to step 8.9; the target object including a chopping board includes targets of chopping boards of M2 colors; the target object including a knife includes targets of knives with handles of M2 colors; Example 8, when q=1, n=6, the identifier CompISym(1,6) of the target object (6) is the identifier of the target object including a person, and the process goes to step 8.3; Example 9, when q=2, n=8, the identifier CompISym(2,8) of the target object (8) is the identifier of the target object including a person, and the process goes to step 8.3; Example 10, when q=2, n=9, the identifier CompISym(2,9) of the target object (9) is the identifier of the target object including a person, and the process goes to step 8.3; Example 11, when q=2, n=4, the identifier CompISym(2,4) of the target object (4) is the identifier of the target object including the tool, and the process goes to step 8.3; Example 11, when q=2, n=5, the identifier of the target object (5) is the identifier of the target object including the cutting board in CompISym (2,5), and go to step 8.3; Step 8.3, obtaining an image ImgP(n) from the image according to the rectangle CompPRect(q,n); the image ImgP(n) is an image with a width of w and a height of h; the rectangle CompPRect(q,n) is the minimum circumscribed rectangle that frames the target object (n) in the identification set IdenSet(q); Embodiment 8, when q=1, n=6, obtaining image ImgP(6) from image Pic(1) according to rectangle CompPRect(1,6); Embodiment 9, when q=2, n=8, obtaining image ImgP(8) from image Pic(2) according to rectangle CompPRect(2,8); Embodiment 10, when q=2, n=9, obtaining image ImgP(9) from the image Pic(2) according to rectangle CompPRect(2,9); Embodiment 11, when q=2, n=4, obtaining image ImgP(4) from the image Pic(2) according to rectangle CompPRect(2,4); Embodiment 11, when q=2, n=5, obtaining image ImgP(5) from the image Pic(2) according to rectangle CompPRect(2,5); Step 8.4, using the neural network to identify the image ImgP(n), and obtaining the identifier IdenSym(q,n) of the image ImgP(n); Example 8, when q=1, n=6, the Yolov8 neural network is used to recognize the image ImgP(6), and the identifier IdenSym(1,6)=21 (a person using a mobile phone) is obtained; Example 9, when q=2, n=8, the Yolov8 neural network is used to recognize the image ImgP(8), and the identifier IdenSym(2,8)=23 (a running person) is obtained; Example 10, when q=2, n=9, the Yolov8 neural network is used to recognize the image ImgP(9), and the identifier IdenSym(2,9)=12 is obtained; Example 11, when q=2, n=4, the Yolov8 neural network is used to recognize the image ImgP(4), and the identifier IdenSym(2,4)=42 is obtained; Example 11, when q=2, n=5, the Yolov8 neural network is used to recognize the image ImgP(5), and the identifier IdenSym(2,5)=36 is obtained; Step 8.5: If the identifier of the target object (n) is an identifier of a target object including a person, go to step 8.6; otherwise, go to step 8.8; Example 8, when n=6, the identifier of the target object (6) is the identifier of the target object including a person, and the process goes to step 8.6; Example 9, when n=8, the identifier of the target object (8) is the identifier of the target object including a person, and the process goes to step 8.6; Example 10, when n=9, the identifier of the target object (9) is the identifier of the target object including a person, and the process goes to step 8.6; Example 11, when n=4, the identifier of the target object (4) is the identifier of the target object including the tool, and the process goes to step 8.8; Example 11, when n=5, the identifier of the target object (5) is the identifier of the target object including the cutting board, and the process goes to step 8.8; Step 8.6, add 1 to the n1; Embodiment 8, when n=6, add 1 to the n1; after the addition, n1=1; Embodiment 9, when n=8, add 1 to the n1; after the addition, n1=1; Embodiment 10, when n=9, the n1 is increased by 1; after the addition, n1=2; Step 8.7. Add the identifier IdenSym(q,n) to the identifier set with the name IdenPeopleSym(q,n1) IdenPeopleSymSet(q); After this step is completed, go to step 8.9; Example 8, when q=1, n=6, n1=1, the identifier IdenSym(1,6)=21 is added to the identification symbol set IdenPeopleSymSet(1) with the name IdenPeopleSym(1,1); after this step is completed, go to step 8.9; Example 9, when q=2, n=8, n1=1, the identifier IdenSym(2,8)=23 is added to the identification symbol set IdenPeopleSymSet(2) with the name IdenPeopleSym(2,1); after this step is completed, go to step 8.9; Example 10, when q=2, n=9, n1=2, the identifier IdenSym(2,9)=12 is added to the identification symbol set IdenPeopleSymSet(2) with the name IdenPeopleSym(2,2); after this step is completed, go to step 8.9; Step 8.8, obtain the knife plate position identifier BdAndKnPlace(q); The steps to obtain the blade position identifier BdAndKnPlace(q) are as follows: Step 8.8.1: If the target object corresponding to the identifier IdenSym(q,n) is not within the range box of the target object, go to step 8.8.2; otherwise, go to step 8.8.3; Example 11, when q=2, n=4, the tool target (4) corresponding to the identifier IdenSym(2,4) is not within the range box LimitR(2,3) of the target (4), and go to step 8.8.2; Example 11, when q=2, n=5, the cutting board object (5) corresponding to the identifier IdenSym(2,5) is within the range box of the object (5), and go to step 8.8.3; Step 8.8.2, set the knife position identifier BdAndKnPlace(q)=false; Example 11, when q=2, n=4, set the blade position identifier BdAndKnPlace(2)=false; Step 8.8.3, end the process of obtaining the blade position identifier BdAndKnPlace(q); In this embodiment, when q=2, the process of obtaining the blade position identifier BdAndKnPlace(q) is terminated; wherein BdAndKnPlace(2)=false; Step 8.9, add 1 to the n; In Example 8 to Example 11, the n is increased by 1; Step 8.10: If n>Nc(q), go to step 8.11; otherwise go to step 8.2; wherein Nc(q) is the number of targets in the identification set IdenSet(q); In this embodiment, if n>Nc(q), go to step 8.11; otherwise go to step 8.2; Step 8.11, end the processing of obtaining the identification symbol set IdenPeopleSymSet(q) and the blade position identifier BdAndKnPlace(q) according to the identification set IdenSet(q).

[0047] Example 8: At the end of this step, IdenPeopleSymSet(1)={IdenPeopleSym(1,1)} and the blade position identifier BdAndKnPlace(1) are obtained; wherein, IdenPeopleSym(1,1)=21; BdAndKnPlace(1)=true.

[0048] Example 10. At the end of this step, IdenPeopleSymSet(2)={IdenPeopleSym(2,n1)|n1=1,2} and the blade position identifier BdAndKnPlace(2) are obtained; among which, IdenPeopleSym(2,1)=23, IdenPeopleSym(2,2)=12; BdAndKnPlace(2)=true.

[0049] Example 11, at the end of this step, IdenPeopleSymSet(2) is obtained as an empty set and the blade position identifier BdAndKnPlace(2)=false is obtained.

[0050] In Embodiment 8 to Embodiment 10, the steps of obtaining the image ImgP(n) from the image according to the rectangle CompPRect(q,n) are as follows: Step 9.1, using formula (1) to calculate the left boundary value PSx(n), the right boundary value PEx(n), the upper boundary value PSy(n), and the lower boundary value PEy(n) of the rectangle CompPRect(q,n) in the rectangle PREct(n) of the image, PSx(n)=int[CPsx(n)×W(q) / w], PEx(n)=int[CPex(n)×W(q) / w] PSy(n)=int[CPsy(n)×H(q) / h], PEy(n)=int[CPey(n)×H(q) / h](1) (1) In the formula, CPsx(n), CPex(n), CPsy(n), and CPey(n) are the left boundary value, right boundary value, upper boundary value, and lower boundary value of the rectangle CompPRect(q,n), respectively; W(q) and H(q) are the width and height of the image, respectively; int[] is an integer operation; wherein the rectangle CompPRect(i) is the minimum bounding rectangle of the compressed image CompP(q) that encloses the target object (n); Embodiment 8, when q=1, n=6, the left boundary value PSx(6), the right boundary value PEx(6), the upper boundary value PSy(6), and the lower boundary value PEy(6) of the rectangle CompPRect(1,6) in the rectangle PREct(6) of the image Pic(1) are calculated using formula (1); W(1) and H(1) are the width and height of the image Pic(1) respectively; Embodiment 9, when q=2, n=8, the left boundary value PSx(8), the right boundary value PEx(8), the upper boundary value PSy(8), and the lower boundary value PEy(8) of the rectangle CompPRect(2,8) in the rectangle PREct(8) of the image Pic(2) are calculated using formula (1); W(2) and H(2) are the width and height of the image Pic(2) respectively; Embodiment 10, when q=2, n=9, the left boundary value PSx(9), the right boundary value PEx(9), the upper boundary value PSy(9), and the lower boundary value PEy(9) of the rectangle CompPRect(2,9) in the rectangle PREct(9) of the image Pic(2) are calculated using formula (1); Embodiment 11, when q=2, n=4, the left boundary value PSx(4), the right boundary value PEx(4), the upper boundary value PSy(4), and the lower boundary value PEy(4) of the rectangle CompPRect(2,4) in the rectangle PREct(4) of the image Pic(2) are calculated using formula (1); Embodiment 11, when q=2, n=5, the left boundary value PSx(5), the right boundary value PEx(5), the upper boundary value PSy(5), and the lower boundary value PEy(5) of the rectangle CompPRect(2,5) in the rectangle PREct(5) of the image Pic(2) are calculated using formula (1); Step 9.2, use formula (2) to calculate the width Pw(n) and height Ph(n) of the rectangle PREct(n), Pw(n)=PEx(n)-PSx(n)+1 Ph(n)=PEy(n)-PSy(n)+1 (2) Embodiment 8, when n=6, the width Pw(6) and the height Ph(6) of the rectangle PREct(6) are calculated using formula (2); Embodiment 9, when n=8, the width Pw(8) and the height Ph(8) of the rectangle PREct(8) are calculated using formula (2); Embodiment 10, when n=9, the width Pw(9) and the height Ph(9) of the rectangle PREct(9) are calculated using formula (2); Embodiment 11, when n=4, the width Pw(4) and the height Ph(4) of the rectangle PREct(4) are calculated using formula (2); Embodiment 11, when n=5, the width Pw(5) and the height Ph(5) of the rectangle PREct(5) are calculated using formula (2); Step 9.3: If Pw(n)≤w∧Ph(n)≤h, go to step 9.4; otherwise go to step 9.5; Example 8, Pw(6)≤w∧Ph(6)≤h, go to step 9.4; Example 9, Pw(8)≤w, Ph(8)>h, go to step 9.5; Example 10, Pw(9)≤w∧Pw(9)≤h, go to step 9.4; Example 11, Pw(4)≤w∧Ph(4)≤h, go to step 9.4; Example 11, Pw(5)≤w∧Pw(5)≤h, go to step 9.4; Step 9.4, according to the boundary value of PREct(n), obtain the image ImgP(n) containing the entire rectangle PREct(n) from the image; after this step is completed, go to step 9.7; the image ImgP(n) is an image with a width of w and a height of h; Embodiment 8, when q=1, n=6, according to the boundary value of PREct(6), an image ImgP(6) including the entire rectangle PREct(6) is obtained from the image Pic(1) with the center point PREctC(6) of PREct(6) as the center; after this step is completed, proceed to step 9.7; as shown in the attached figure Figure 7 As shown; Embodiment 10, when q=2, n=9, according to the boundary value of PREct(9), an image ImgP(9) including the entire rectangle PREct(9) is obtained from the image Pic(2) with the center point PREctC(9) of PREct(9) as the center; after this step is completed, proceed to step 9.7; Embodiment 11, when q=2, n=4, according to the boundary value of PREct(4), an image ImgP(4) including the entire rectangle PREct(4) is obtained from the image Pic(2) with the center point PREctC(4) of PREct(4) as the center; after this step is completed, go to step 9.7; Embodiment 11, when q=2, n=5, according to the boundary value of PREct(5), an image ImgP(5) including the entire rectangle PREct(5) is obtained from the image Pic(2) with the center point PREctC(5) of PREct(5) as the center; after this step is completed, go to step 9.7; Step 9.5, according to the boundary value of PREct(n), obtain the image EImgP(n) containing the entire rectangle PREct(n) from the image; the width of EImgP(n) is EPw(n) and the height is EpH(n); EPw(n) and EpH(n) are calculated by formula (3), EPw(n)=Pw(n),EPh(n)=int[h×Pw(n) / w] if Pw(n) / w≥Ph(n) / h EPw(n)=int[w×Ph(n) / h],EPh(n)=Ph(n) else (3) Embodiment 9, when q=2, n=8, according to the boundary value of PREct(8), an image EImgP(8) including the entire rectangle PREct(8) is obtained from the image Pic(2) with the center point PREctC(8) of PREct(8) as the center; the width of EImgP(8) is EPw(8) and the height is EpH(8); EPw(8) and EpH(8) are calculated by formula (4), because Ph(8) / h>Pw(8) / w, so, EPw(8)=int[w×Ph(8) / h], EPh(8)=Ph(8) (3) Step 9.6, compress the image EImgP(n) into an image ImgP(n) with a width of w and a height of h; Embodiment 9, when n=8, compress the image EImgP(8) into an image ImgP(8) with a width of w and a height of h; as shown in the attached figure Figure 8 As shown; Step 9.7, end the process of obtaining the image ImgP(n) from the image according to the rectangle CompPRect(q,n).

[0051] In this embodiment, the process of obtaining the image ImgP(n) from the image according to the rectangle CompPRect(q,n) is ended.

[0052] In Example 3 to Example 11, the steps of event processing are as follows: Generate event records; the generating of event records refers to the intelligent analyzer generating an event record set; the event record set includes at least the following elements: event name, time of event occurrence, location of event occurrence, event type, picture reflecting the event, or also includes GIF animation, or also includes short video, or also includes video recording reflecting the event; In this embodiment, the event record set includes the following elements: event name, time of event occurrence, location of event occurrence, event type, picture reflecting the event, GIF animation, short video, video reflecting the event, the GIF animation refers to an animation reflecting the event content composed of RS pictures, 5≤RS≤20, the short video is a short video reflecting the event content with a duration of SV seconds, 1s≤SV≤2s, the video recording reflecting the event process is a video recording with a length of SumTime seconds, and the format of the short video and the format of the video are both one of the following formats: avi, wmv, mpeg, mp4; where SumTime=Time1+EventTLeng+Time2, Time1 is the duration before the event is recorded, 3s≤Time1≤5s, EventTLeng is the duration of the event, 5s≤EventTLeng≤60s, Time2 is the duration recorded after the duration EventTLeng, 3s≤Time2≤5s. In this embodiment, RS=20, SV=2s, EventTLeng=50s, Time1=Time2=5s. The formats of short videos and videos are both avi. Example 3: The intelligent analyzer generates an event record set, which includes the following elements: event name: smoke, time of event occurrence: 2024 / 06 / 05 / 12:35:07, location of event occurrence: Second Canteen of A1 Middle School, X1 City, event type: smoke, pictures reflecting the event are pictures taken at 2024 / 06 / 05 / 12:35:07, a 20-picture animated picture reflecting the content of the event, a 2-second short video reflecting the content of the event, and a 60-second video recording reflecting the event process; wherein, Time1=Time2=5s, EventTLeng=50s (the duration of the smoke is ≥50s); Example 4: The intelligent analyzer generates an event record set, which includes the following elements: event name: fire, time of event occurrence: 2024 / 06 / 07 / 08:32:09, location of event occurrence: First Canteen of A2 University, X1 City, event type: fire, pictures reflecting the event are pictures taken at 2024 / 06 / 07 / 08:32:09, a 20-picture animated picture reflecting the event content, a 2-second short video reflecting the event content, and a 33-second video recording reflecting the event process; wherein, Time1=Time2=5s, EventTLeng=23s (the duration of the fire is 23s); Example 5, the intelligent analyzer generates an event record set, the event record set includes the following elements: event name: ground with waste, time of event occurrence: 2024 / 06 / 08 / 16:39:55, location of event occurrence: first canteen of B2 Middle School in Y1 City, event type: ground with waste, pictures reflecting the event are pictures taken at 2024 / 06 / 08 / 16:39:55, a 20-picture animated picture reflecting the content of the event, a short video reflecting the content of the event with a duration of 2 seconds, and a video recording reflecting the event process is a video recording with a length of 60 seconds; wherein, Time1=Time2=5s, EventTLeng=50s (the duration of the ground with waste ≥50s); Example 6, the intelligent analyzer generates an event record set, the event record set includes the following elements: event name: incorrect position of food basket, time of event occurrence: 2024 / 05 / 22 / 07:26:12, location of event occurrence: second canteen of B1 University, Y2 City, event type: incorrect position of food basket, picture reflecting the event is a picture taken at 2024 / 05 / 22 / 07:26:12, a 20-picture animated picture reflecting the content of the event, a 2-second short video reflecting the content of the event, and a video recording reflecting the event process is a 60-second video recording; wherein, Time1=Time2=5s, EventTLeng=50s (the duration of incorrect position of food basket ≥50s); Example 7, the intelligent analyzer generates an event record set, the event record set includes the following elements: event name: small animals, time of event occurrence: 2024 / 05 / 20 / 23:15:29, location of event occurrence: first canteen of C2 University, Z1 City, event type: small animals, pictures reflecting the event are pictures taken at 024 / 05 / 20 / 23:15:29, a 20-picture animated picture reflecting the content of the event, a short video reflecting the content of the event with a duration of 2 seconds, and a video recording reflecting the event process is a video recording with a length of 17 seconds; wherein, Time1=Time2=5s, EventTLeng=7s (duration of small animals=7s); Example 8: The intelligent analyzer generates an event record set, which includes the following elements: event name: person smoking, time of event occurrence: 2024 / 05 / 30 / 10:27:33, location of event occurrence: the third canteen of C2 University, Z1 City, event type: person smoking, a picture reflecting the event is a picture taken at 2024 / 05 / 30 / 10:27:33, a 20-picture animated picture reflecting the event content, a 2-second short video reflecting the event content, and a video recording reflecting the event process is a 28-second video recording; wherein, Time1=Time2=5s, EventTLeng=18s (duration of smoking=18s); Example 9: The intelligent analyzer generates an event record set, which includes the following elements: event name: person running, time of event occurrence: 2024 / 05 / 27 / 12:39:22, location of event occurrence: the second canteen of E1 Middle School in Z1 City, event type: person running, pictures reflecting the event are pictures taken at 2024 / 05 / 27 / 12:39:22, a 20-picture animated picture reflecting the content of the event, a short video reflecting the content of the event with a duration of 2 seconds, and a video recording reflecting the event process is a video recording with a length of 15 seconds; wherein, Time1=Time2=5s, EventTLeng=5s (the duration of the person running ActTime=5s); Example 11, the intelligent analyzer generates an event record set, the event record set includes the following elements: event name: incorrect tool position, time of event occurrence: 2024 / 05 / 31 / 12:29:09, location of event occurrence: first canteen of E2 Middle School, Z1 City, event type: incorrect tool position, the picture reflecting the event is a picture taken at 2024 / 05 / 31 / 12:29:09, a 20-picture animated picture reflecting the event content, a 2-second short video reflecting the event content, and a 60-second video recording reflecting the event process; wherein, Time1=Time2=5s, EventTLeng=50s (the duration of incorrect tool position ≥50s); Uploading event records; the uploading event records refers to the intelligent analyzer uploading the event record set to the control server through the switch, network equipment, network, and system switch; In this embodiment, the intelligent analyzer uploads the event record set to the control server through the switch, network equipment, network, and system switch; Recording events: recording events means that the control server saves the event record set to the storage device of the control server; In this embodiment, the control server saves the event record set to the hard disk of the control server; Event record publishing; event record publishing means that the control server publishes the event record set to the client with the permission to view the event record set and the mobile phone with the permission to view the event record set through the system switch and the network; In Example 3, the control server publishes the event record set to the client and mobile terminal that have the authority to view the event record set of the second canteen of A1 Middle School in X1 City through the system switch and the network; In Example 4, the control server publishes the event record set to the client and mobile terminal that have the authority to view the event record set of the first canteen of A2 University in X1 City through the system switch and the network; Event alarm processing; the event alarm processing means that if the event is judged to be a smoke event, an alarm notification is made; if the event is judged to be a fire event, an alarm notification and event transmission are made; the alarm notification means that the control server sends the event information of smoke or fire to the relevant mobile phones and clients through the system switch and network; the event transmission means that the control server transmits the event record set to the system set place in the external network through the system switch, network, and network device Wn.

[0053] In Example 3, the control server sends smoke event information to the mobile phone terminal and client terminal related to the second canteen of A1 Middle School in X1 City through the system switch and the network.

[0054] In Example 4, the control server sends the fire event information to the mobile phone and client related to the first canteen of A2 University in X1 City through the system switch and the network, and the control server transmits the event record set to the fire center set by the system in the external network or the management center that manages the first canteen of A2 University in X1 City through the system switch, network, and network device Wn.

[0055] The effect of the above technical solution is: using Yolov8 neural network to identify the target object and obtain the identification set, and then based on the identification set, it can monitor events such as smoke, fire, and incorrect position of the food basket; when a person or a cutting board or a knife is detected, according to the minimum circumscribed rectangle of the target object containing the person or the cutting board or the knife in the compressed image, an image ImgP(n) containing the entire target object with a width of w and a height of h is intercepted from the original image, and then the Yolov8 neural network is used to identify this image ImgP(n), and an identification symbol set and a knife board position identification symbol are obtained, and then it is determined whether there is a person who does not meet the specifications or the cutting board position is incorrect or the knife position is incorrect; this technical solution also uses a difference map to monitor moving objects, and when the moving objects are monitored After that, an image with a width w and a height h framing the moving object is obtained from the current frame image, and the Yolov8 neural network is used to recognize the image, thereby detecting small animals. This scheme has a high monitoring sensitivity. When recognizing and processing at the resolution of the original image, the neural network can accurately identify events such as people who do not meet the specifications and the appearance of small animals, and can also recognize the color of the side of the chopping board and the color of the knife handle (the side of the chopping board and the knife handle are small in size and cannot be recognized or cannot be accurately recognized in the compressed image), and then monitor the corresponding events. It can be seen that the reliability, accuracy, extensiveness and practicality of this technical scheme are better than those of the existing technical schemes. This technical scheme uses the Yolov8 neural network to process the width w and height h of the image. are both equal to 608. Such w and h can reliably identify fire, waste and other objects with a width ≥ 83 mm and a height ≥ 83 mm within the range monitored by the camera (such as a width of 10m and a height of 10m). Compared with the prior art that uses other neural networks or other methods to identify smaller objects, the reliability and accuracy of identification are higher. In addition, compared with the prior art that uses a w*h image (a w*h image is an image with a width of w and a height of h) to cover and scan the original image to find the target object, the technical solution has higher recognition efficiency, better recognition accuracy and more complete identified targets. This is because the prior art usually overlaps the w*h image in the X and Y directions in order not to segment the target object during scanning. Scanning (e.g., the w*h image on the right overlaps 1 / 2 of the w*h image on the left in the X direction, and the w*h image below overlaps 1 / 2 of the w*h image above in the Y direction), which not only increases the number of scans, but also reduces the efficiency and real-time performance of recognition, and it is also impossible to guarantee that an image containing a complete target object (e.g., a target object at the boundary of the overlapping area) can be obtained. Another drawback of the prior art method of using a w*h image to cover scan the original image to find the target object is that if the target object is close to the camera, the width of the target object may exceed w, or the height of the target object may exceed h, so that the acquired w*h image may not capture the complete target object, which reduces the reliability, integrity, and accuracy of recognition.When this situation occurs, the technical solution first extracts the image containing the complete target object (such as EImgP(n)) from the image, and then compresses the image containing the complete target object (such as EImgP(n)) into a w*h image ImgP(n), and then uses the Yolov8 neural network to recognize the w*h image ImgP(n); this not only ensures the integrity, accuracy and reliability of the recognition, but also greatly improves the efficiency of processing and recognition, making the system more practical, reliable, accurate and real-time. ;

[0056] In Example 1 and Example 2, the steps of generating the neural network are as follows: Step 11.1, set the number of acquired pictures PicNum, and set the following sets as empty sets: atlas, preprocessing set, compression set, atlas of set target objects, processing set of set target objects, processing set; wherein PicNum is a positive integer ≥ 3000; the set targets include at least target 8, target 9, ..., target 26, targets of cutting boards of M2 colors, and targets of knives with knife handles of M2 colors; In this embodiment, the number of acquired pictures PicNum is set to 50000, and the following sets are set to empty sets: atlas PicSet, preprocessing set PreProcSet, compression set CompSet, atlas AppointImgSet for setting target objects, processing set AppointProcSet for setting target objects, and processing set ProcSet; the target objects are set to include at least target object 8, target object 9, ..., target object 26, target object 35, target object 36, ..., target object 48; Step 11.2, obtain the atlas PicSet; wherein PicSet={P(j)|j=1,2,...,SumJ}; P(j) is a picture; j is the number of the picture in the order of time when it is added to the atlas PicSet; SumJ is the number of pictures in the atlas PicSet; Example 1, SumJ=56380; Example 2, SumJ=69270; Step 11.3, use the atlas PicSet to obtain the processing set ProcSet; wherein, ProcSet={CompP(aj), CompPFile(aj)|aj=1,2,...,J}; wherein, CompP(aj) is a picture with a width of w and a height of h; CompPFile(aj) is the annotation file of the picture CompP(aj); J is the number of pictures in the processing set ProcSet, J=SumJ+PV; PV is the number of pictures in the processing set AppointProcSet of the set target object, PV is a positive integer; Example 1, J=SumJ+PV=78950; wherein, SumJ=56380, PV=22570; Example 2, J=SumJ+PV=104760; wherein, SumJ=69270, PV=35490; Step 11.4, split the processing set ProcSet into a training set TrainSet and a test set TestSet; wherein TrainSet={CompP(r), CompPFile(r)|r=1,2,...,R}, R=0.6×J; TestSet={CompP(g),CompPFile(g)|g=1,2,...,G}, G=0.4×J; Step 11.5, training the neural network model; training the neural network model means training the neural network model with each set of pictures and files in the training set; a set of pictures and files means a picture and a labeling file of the picture; in this embodiment, the neural network model means the Yolov8 neural network model; The specific steps for training a neural network model are as follows: Step 11.5.1, set the initial value of r to 1, set TrainPn to the number of images in the training set TrainSet; r is the number of the image and annotation file in a set of images and annotation files in the training set TrainSet; Example 1, set r=1, set TrainPn=47370; Example 2: set r=1, set TrainPn=62856; Step 11.5.2, the training neural network model program in the GPU server uses the picture CompP(r) and the annotation file CompPFile(r) in TrainSet to train the Yolov8 neural network model; Step 11.5.3, add 1 to the r; Step 11.5.4: If r>TrainPn, go to step 11.5.5; otherwise go to step 11.5.2; Step 11.5.5, end the process of training the neural network model; Step 11.6, testing the neural network model; testing the neural network model means testing the neural network model with each set of images and files in the test set, and counting the number of each target object tested and the number of each target object tested correctly; The specific steps for testing the neural network model are as follows: Step 11.6.1, set the initial value of g to 1, set TestPn to the number of images in the test set TestSet, set the initial value of each SumTN(k) to 0, and the initial value of each SumCN(k) to 0, k=1,2,...,K; where g is the number of images and annotation files in a set of images and annotation files in the test set; SumTN(k) is the number of tests for target object k; SumCN(k) is the number of correct tests for target object k; In Example 1, g=1 is set, TestPn=31580 is set, each SumTN(k)=0, each SumCN(k)=0, k=1,2,...,48; In Example 2, g=1 is set, TestPn=41904 is set, each SumTN(k)=0, each SumCN(k)=0, k=1,2,...,48; Step 11.6.2, the test neural network model program in the GPU server uses the image CompP(g) in the test set TestSet and the annotation file CompPFile(g) to test the Yolov8 neural network model, and counts each TestNum(k) and each CorrectNum(k), k=1,2,...,K; wherein, TestNum(k) is the number of target objects k in the annotation file CompPFile(g); CorrectNum(k) is the number of target objects k tested correctly; a target object (m) in the annotation file CompPFile(g) that is a target object k is tested correctly, which means that the Yolov8 neural network model's recognition of the target object (m) in the test satisfies the following three conditions at the same time: ① Identify the target object ( m); ② The rectangle RegRect(m) of the identified target object (m) satisfies the condition [Pn(RegRect(m)∩CompRect(m)) / Pn(CompRect(m))]×100%≥MinSv; where Pn(AR) is the number of pixels in the area AR; 80%≤MinSv≤100%; CompRect(m) is the minimum bounding rectangle that frames the target object (m) in the annotation file CompPFile(g); ③ The identifier RegSym(m)=CLabSym(m) of the target object (m); RegSym(m) is the marker of the target object (m) identified by the Yolov8 neural network model; CLabSym(m) is the marker of the target object (m) in the annotation file CompPFile(s) = the marker of the target object k; In this embodiment, when g=1, the test neural network model program in the GPU server uses the image CompP(1) in the test set TestSet and the annotation file CompPFile(1) to test the Yolov8 neural network model, and statistics are given as TestNum(k)=1, CorrectNum(k)=1, k=1, 2, ..., 7; TestNum(12)=2, CorrectNum(12)=2; TestNum(k)=0, CorrectNum(k)=0, k=8, 9, 10, 11, 13, 14, ..., 48; the target object (1) in the annotation file CompPFile(1) is the target object 12. Correct test means that the Yolov8 neural network model meets the following three conditions for the recognition of the target object (1) in the test: ① From the image ① The object (1) is identified in the image CompP(1); ② The rectangle RegRect(1) of the identified object (1) satisfies the condition [Pn(RegRect(1)∩CompRect(1)) / Pn(CompRect(1))]×100%≥85%; wherein, MinSv=85%; CompRect(1) is the minimum enclosing rectangle that frames the object (1) in the annotation file CompPFile(1); ③ The identifier RegSym(1) of the object (1)=CLabSym(1); CLabSym(1) is the marker of the object (1) in the annotation file CompPFile(1)=the marker of the object 12=12; similarly, another object (2) in the annotation file CompPFile(1) which is the object 12 is also tested correctly; Step 11.6.3, accumulate each SumCN(k) and accumulate each SumTN(k), k=1,2,...,K; In Example 1 and Example 2, each SumCN(k)=SumCN(k)+CorrectNum(k) is accumulated, and each SumTN(k)=SumTN(k)+TestNum(k) is accumulated, k=1,2,...,48; wherein SumCN(k) and SumTN(k) on the left side of the formula are the accumulated values, and SumCN(k) and SumTN(k) on the right side of the formula are the values ​​before the accumulation; Example 1, when g=1, SumCN(k)=1, SumTN(k)=1, k=1, 2, ..., 7; SumCN(12)=2, SumTN(12)=2; SumCN(k)=0, SumTN(k)=0, k=8, 9, 10, 11, 13, 14, ..., 48; Step 11.6.4, add 1 to the g; Step 11.6.5: If g>TestPn, go to step 11.6.6; otherwise go to step 11.6.2; Step 11.6.6, end the processing of the test neural network model; Step 11.7, count the test accuracy of each target object k, k=1,2,...,K; In Example 1 and Example 2, the test accuracy rate Av(k) of each target object k is counted, k=1, 2, ..., 48; wherein Av(k)=[SumCN(k) / SumTN(k)]×100%; Step 11.8: If the test accuracy Av(k) of each target object k is ≥ SetAv, go to step 11.10; otherwise, go to step 11.9; where 85%≤SetAv≤98%; Example 1, the test accuracy of target object 27 Av(27)=90%<95%, go to step 11.9; in this example, SetAv=95%; Example 2, the recognition accuracy Av(k) of each target object k is ≥95%, k=1, 2, ..., 48, then go to step 11.10; Step 11.9, increase the PicNum value, and then go to step 11.2; increasing the PicNum value means that the PicNum after increase = PicNum before increase × ChgScaleV; wherein, 1.1≤ChgScaleV≤2.4; Embodiment 1, increase the PicNum value, and after the increase, go to step 11.2; increasing the PicNum value means that after the increase, PicNum=PicNum before the increase×ChgScaleV; after the increase, PicNum=60000; wherein, ChgScaleV=1.2; Step 11.10, compiling the neural network model into a neural network; the neural network is a neural network that can identify the target object; In embodiment 2, the GPU server compiles the Yolov8 neural network model into a Yolov8 neural network using a compiling software; the Yolov8 neural network refers to a Yolov8 neural network that can identify the target object; Step 11.11, writing the neural network into each intelligent analyzer in the system; In Example 2, the server takes out the Yolov8 neural network in the GPU server through the system switch, and writes the Yolov8 neural network into each intelligent analyzer in the system through the system switch, the network and the network devices and switches in each of the 80 monitoring subsystems; Step 11.12, end the process of generating the neural network.

[0057] Example 2, ending the process of generating the neural network; in this example, the completion time of generating the neural network is 2024 / 05 / 08 / 15:20.

[0058] In Example 1 and Example 2, the steps of obtaining the atlas are as follows: Obtain images containing target objects from video recordings; The steps to obtain a picture containing the target object from a video recording are as follows: Obtaining video; obtaining video means obtaining D1 video segments containing the target object; D1 is a positive integer; In this embodiment, the control server uses the video downloading software to download D1 segments of video recordings d1 containing the target object in different time periods from each of the 10 NVRs connected thereto, where d1=StrDx,StrDx+1,...,D1; wherein d1 is the number of the video recording segments generated in the order of the time when the video recording segments are downloaded to the control server; D1 is the number of the downloaded video recording segments; the duration of each video recording segment is 1 hour; StrDx is the starting number of the downloaded video recording segment; Embodiment 1, StrDx=1, D1=420; Embodiment 2, StrDx=421, D1=500; Get the picture; Get the picture means to intercept the picture from each video recording d1, d1=StrDx,StrDx+1,...,D1, and add the intercepted picture to the temporary storage set TmpSet(d), d=StrD,StrD+1,...,D; where d1 is the number of the video recording segment without intercepted pictures; StrDx is the starting number of the video recording segment without intercepted pictures; d is the number of the temporary storage set; StrD is the starting number of the temporary storage set; D is the number of the temporary storage set, and D is a positive integer; In this embodiment, the steps of obtaining the picture are as follows: Step 12.2.1, set the initial value of d1 to StrDx, and set the initial value of d to StrD; In Example 1, d1=StrDx=1 is set, and d=StrD=1 is set; In Example 2, d1=StrDx=421 and d=StrD=421 are set; Step 12.2.2, set the temporary set TmpSet(d) to an empty set; Step 12.2.3, use the screenshot software to capture the picture Pt(d,u) from the video recording d1, and add the picture Pt(d,u) to the temporary storage set TmpSet(d); wherein, TmpSet(d)={Pt(d,u)|u=1,2,...,U(d)}; u is the number of the pictures in the temporary storage set TmpSet(d) in the order of the time of adding to the temporary storage set TmpSet(d); U(d) is the number of pictures in the temporary storage set TmpSet(d), and U(d) is a positive integer; the resolution of the picture Pt(d,u) is the resolution of the video recording; the time interval for the screenshot software to capture the picture from the video recording d1 is TimeS seconds; 10s≤TimeS≤30s; in this embodiment, TimeS=20s; Step 12.2.4, add 1 to the d1, and add 1 to the d; Step 12.2.5: If d1>D1, go to step 12.2.6; otherwise go to step 12.2.2; Step 12.2.6, end the process of obtaining the picture; At the end of this step, TmpSet(d)={Pt(d,u)|u=1,2,...,U(d)}, d=1,2,...,D; where Embodiment 1, D=420; Embodiment 2, D=500; Step 12.3, filter the image.

[0059] In this embodiment, the specific steps of filtering pictures are as follows: Step 12.3.1, delete the same pictures; To delete the same pictures, follow these steps: Step 12.3.1.1, set the initial value of d to StrD; d is the number of the temporary storage set TmpSet(d) of the same picture that has not been deleted; StrD is the starting number of the temporary storage set TmpSet(d) of the same picture that has not been deleted; In Example 1, set d=StrD=1; Example 2, set d=StrD=421; Step 12.3.1.2, delete the same pictures in the temporary set TmpSet(d); The steps to delete the same pictures in the temporary set TmpSet(d) are as follows: Step 12.3.1.2.1, set the position of the image Pt(d,u) observed by the image management software to the position of Pt(d,1) where u=1; in this embodiment, the image management software is the image management software ACDSee; the images are all images in the temporary storage set TmpSet(d); In this embodiment, when d=1 and u=1, the position of the image Pt(1,u) observed by the image management software ACDSee is set to the position of Pt(1,1) where u=1; Step 12.3.1.2.2, use the picture management software to observe the picture Pt(d,u) and the picture Pt(d,u+1), if the picture Pt(d,u) and the picture Pt(d,u+1) are different pictures, go to step 12.3.1.2.3; otherwise, go to step 12.3.1.2.4; In this embodiment, when d=1 and u=1, the picture management software ACDSee is used to observe the picture Pt(1,1) and the picture Pt(1,2). The picture Pt(1,1) and the picture Pt(1,2) are different pictures, and the process goes to step 12.3.1.2.3. The reason why the picture Pt(1,1) and the picture Pt(1,2) are different pictures is that the picture Pt(1,1) and the picture Pt(1,2) have two different targets. In this embodiment, when d=1, u=2, similar observation is made and it is observed that the picture Pt(1,2) and the picture Pt(1,3) are the same picture, and the process goes to step 12.3.1.2.4; In this embodiment, when d=1 and u=5, similar observation is made and it is observed that the picture Pt(1,5) and the picture Pt(1,6) are different pictures, and the process goes to step 12.3.1.2.3; the picture Pt(1,5) and the picture Pt(1,6) are different pictures because the number of target objects in the picture Pt(1,5) and the picture Pt(1,6) is different; Step 12.3.1.2.3, use the image management software to move the position of the observed image to the position of the new image Pt(d,u); after completing this step, go to step 12.3.1.2.7; where u=Min(U(d),u+1); the left side of the equation is the new u, and the right side of the equation is the original u; Min() is the minimum value operation; In this embodiment, when d=1, u=1, the image management software ACDSee is used to move the position of the observed image to the position of the new image Pt(1,2); after this step is completed, go to step 12.3.1.2.7; wherein u=Min(U(1),u+1)=2; In this embodiment, when d=1 and u=5, the image management software ACDSee is used to move the position of the observed image to the position of the new image Pt(1,6); after this step is completed, go to step 12.3.1.2.7; wherein u=Min(U(1),u+1)=6; Step 12.3.1.2.4, using the image management software to observe the image Pt(d,u) and the subsequent image Pt(d,u+au); observe until the image Pt(d,u) is different from the subsequent image Pt(d,u+au); where au≥2; In this embodiment, when d=1, u=2, au=3, k=12, the picture management software ACDSee is used to observe the picture Pt(1,2) and the subsequent picture Pt(1,2+au); it is observed that the picture Pt(1,2) is the same as the picture Pt(1,4), and it is observed that the picture Pt(1,2) is different from the picture Pt(1,5); wherein au=3; the picture Pt(1,2) is different from the picture Pt(1,5) means that PArea(1,2,12)∩PArea(1,5,12) is an empty set; 2≠5∧6∈Z[1, 48]; where PArea(1,2,12) and PArea(1,5,12) are the area where the target object 12 (person) in the image Pt(1,2) is located in the middle of the image Pt(1,2) and the area where the same target object 12 in the image Pt(1,5) is located in the lower left corner of the image Pt(1,5), respectively, and the operation of PArea(1,2,12)∩PArea(1,5,12) is performed when the images Pt(1,2) and Pt(1,5) are in the same image coordinate system of Pt(1,2); Step 12.3.1.2.5, use the picture management software to delete the picture Pt(d,u+1), the picture Pt(d,u+2), ..., the picture Pt(d,u+au-1); In this embodiment, when d=1, u=2, au=3, the picture management software ACDSee is used to delete the picture Pt(1,3) and the picture Pt(1,4); Step 12.3.1.2.6. Use the image management software to move the position of the observed image to the position of the new image Pt(d,u); where u=Min(U(d),u+au); In this embodiment, when d=1, u=2, au=3, the image management software ACDSee is used to move the position of the observed image to the position of the new image Pt(1,5); wherein u=Min(U(1),u+au)=5; Step 12.3.1.2.7. If u=U(d), go to step 12.3.1.2.8; otherwise go to step 12.3.1.2.2; where U(d) is the number of pictures in the temporary set TmpSet(d); In this embodiment, when d=1, if u=U(1)=178, go to step 12.3.1.2.8; otherwise, go to step 12.3.1.2.2; Step 12.3.1.2.8, end the process of deleting the same pictures in the temporary set TmpSet(d); Step 12.3.1.3, add 1 to the d; Step 12.3.1.4: If d>D, go to step 12.3.1.5; otherwise go to step 12.3.1.2; Step 12.3.1.5, end the process of deleting the same pictures; Step 12.3.2, save the pictures in the temporary atlas to the atlas; The steps to save the images in the temporary atlas to the atlas are as follows: Step 12.3.2.1, set the initial value of d to StrD, and set the initial value of j to StrJ; d is the number of the temporary set that is not saved in the atlas; StrD is the starting number of the temporary set that is not saved in the atlas; j is the number of the picture saved in the atlas; StrJ is the starting number of the picture saved in the atlas; In Example 1, d=StrD=1 is set, and j=StrJ=1 is set; In Example 2, d=StrD=421 is set, and j=StrJ=56381 is set; Step 12.3.2.2, save the pictures in the temporary set TmpSet(d) to the atlas; The steps to save the images in the temporary set TmpSet(d) to the atlas are as follows: Step 12.3.2.2.1, set the initial value of u to 1; u is the number of the picture in the temporary set TmpSet(d); Step 12.3.2.2.2: If the image filtering program determines that the image Pt(d,u) exists in the temporary set TmpSet(d), go to step 12.3.2.2.3; otherwise, go to step 12.3.2.2.5; In this embodiment, when d=1, u=1, j=1, the image filtering program determines that the image Pt(1,1) exists in the temporary set TmpSet(1), and then goes to step 12.3.2.2.3; In this embodiment, when d=1, u=2, j=2, the image filtering program determines that the image Pt(1,2) exists in the temporary set TmpSet(1), and then goes to step 12.3.2.2.3; In this embodiment, when d=1, u=3, u=4, j=3, the image filtering program determines that there is no image Pt(1,3) and no image Pt(1,4) in the temporary set TmpSet(1), and then goes to step 12.3.2.2.5; In this embodiment, when d=1, u=5, j=3, the image filtering program determines that the image Pt(1,5) exists in the temporary set TmpSet(1), and then goes to step 12.3.2.2.3; Step 12.3.2.2.3, the filtering image program saves the image Pt(d,u) to the image set PicSet with the name Pm(j); In this embodiment, when d=1, u=1, j=1, the image filtering program saves the image Pt(1,1) to the image set PicSet with the name Pm(1); In this embodiment, when d=1, u=2, j=2, the image filtering program saves the image Pt(1,2) to the image set PicSet with the name Pm(2); In this embodiment, when d=1, u=5, j=3, the image filtering program saves the image Pt(1,5) to the image set PicSet with the name Pm(3); Step 12.3.2.2.4, add 1 to the j; In this embodiment, when j=1, j=2, and j=3, 1 is added to j; after the addition, j is 2, 3, and 4 respectively; Step 12.3.2.2.5, add 1 to the u; In this embodiment, when u=1, 2, 3, 4, 5, 1 is added to u; after addition, u=2, 3, 4, 5, 6 respectively; Step 12.3.2.2.6: If u>U(d), go to step 12.3.2.2.7; otherwise go to step 12.3.2.2.2; In this embodiment, when d=1, if u>U(1)=178, go to step 12.3.2.2.7; otherwise go to step 12.3.2.2.2; Step 12.3.2.2.7, end the process of saving the pictures in the temporary set TmpSet(d) to the atlas; In this embodiment, when d=1, the process of saving the pictures in the temporary set TmpSet(1) to the atlas ends; Step 12.3.2.3, add 1 to the d; Step 12.3.2.4: If d>D, go to step 12.3.2.5; otherwise go to step 12.3.2.2; Step 12.3.2.5, ending the process of saving the pictures in the temporary atlas to the atlas; Step 12.3.3, end the processing of the filtered image.

[0060] After this step is completed, PicSet={Pm(j)|j=1,2,...,SumJ}; where SumJ is the number of pictures in the atlas PicSet; Example 1, SumJ=56380 (≥PicNum=50000).

[0061] Example 2, SumJ=69270 (≥PicNum=60000).

[0062] In Example 1 and Example 2, the steps of obtaining the processing set ProcSet using the atlas PicSet are as follows: Labeling processing; in this embodiment, the rectangle marking the position of the target object is a rectangle represented by four boundary values; the marker of the target object is a different symbol Symbol(k)=k assigned to different targets k, k=1,2,...,48; In this embodiment, the specific steps of the marking process are as follows: Step 13.1.1, set the initial value of j to StrJ, set the initial value of pj to PStrJ, set PicPn to the number of pictures in the atlas PicSet; j is the number of unlabeled pictures in the atlas PicSet; StrJ is the starting number of unlabeled pictures in the atlas PicSet; pj is the number of pictures in the preprocessing set PreProcSet; PStrJ is the starting number of pictures in the preprocessing set PreProcSet; In Example 1, set j=StrJ=1, set pj=PStrJ=1, and set PicPn=|PicSet|=56380; Example 2, set j=StrJ=56381, set pj=PStrJ=56381, set PicPn=|PicSet|=69720; Step 13.1.2, automatically annotate the picture Pm(j) in the atlas PicSet using automatic annotation software to obtain the annotation file LabPFile(pj) of the picture Pm(j); wherein, LabPFile(pj)={Rect(m),LabSym(m)|m=1,2,...,Mp(pj)}; Rect(m) is a rectangle marking the position of the target object (m), LabSym(m) is a marker of the target object (m), m=1,2,...,Mp(pj); m is a number formed in increasing order of the center point of the rectangle Rect(m); Mp(pj) is the number of the target objects identified from the picture Pm(j), and Mp(pj) is a positive integer; In this embodiment, when j=1 and pj=1, the automatic annotation software is used to automatically annotate the picture Pm(1) in the atlas PicSet to obtain the annotation file LabPFile(1) of the picture Pm(1); wherein LabPFile(1)={Rect(m),LabSym(m)|m=1,2,...,8}; Rect(m) is a rectangle marking the position of the target object (m) (Rect(m) is not necessarily the smallest circumscribed rectangle that frames the target object (m)), LabSym(m) is a marker of the target object (m) (LabSym(m) is not necessarily the correct marker for marking the target object (m)), m=1,2,...,8; Mp(1) is the number of targets identified from the picture Pm(1)=8 (the actual number of targets in the picture Pm(1) is 9); Step 13.1.3, adding the picture Pm(j) to the preprocessing set PreProcSet with the name P(pj), and adding the annotation file LabPFile(pj) to the preprocessing set PreProcSet; In this embodiment, when j=1, pj=1, the picture Pm(1) is added to the preprocessing set PreProcSet with the name P(1), and the annotation file LabPFile(1) is added to the preprocessing set PreProcSet; Step 13.1.4, add 1 to the j, and add 1 to the pj; Step 13.1.5: If j>PicPn, go to step 13.1.6; otherwise go to step 13.1.2; Step 13.1.6, end the marking process; In this embodiment, at the end of this step, PreProcSet={P(pj),LabPFile(pj)|pj=1,2,...,SumJ}; wherein SumJ is the number of images in the preprocessing set PreProcSet; Correction processing; In this embodiment, the specific steps of the correction process are as follows: Step 13.2.1, set the initial value of pj to PStrJ; pj is the number of the uncorrected annotation file in the preprocessing set PreProcSet; PStrJ is the starting number of the uncorrected annotation file in the preprocessing set PreProcSet; In Example 1, pj=PStrJ=1 is set; Example 2: Set pj=PStrJ=56381; Step 13.2.2, using the modified annotation software to judge the image P(pj) in the preprocessing set PreProcSet and the annotation file LabPFile(pj), if it is determined that the annotation file LabPFile(pj) is a correct annotation file, then go to step 13.2.5; otherwise, go to step 13.2.3; in this embodiment, the modified annotation software is the modified annotation software LabelImg; In this embodiment, when pj=1, the modified labeling software LabelImg is used to judge the image P(1) in the preprocessing set PreProcSet and the labeling file LabPFile(1), and it is judged that the labeling file LabPFile(1) is an incorrect labeling file, and then the process goes to step 13.2.3; In this embodiment, when pj=2, it is similarly determined that the labeling file LabPFile(2) is the correct labeling file, and the process goes to step 13.2.5; Step 13.2.3, use the labeling modification software LabelImg to modify the labeling file LabPFile(pj) to be a correct labeling file; the correct labeling file LabPFile(pj) refers to LabPFile(pj)={Rect(m),LabSym(m)|m=1,2,...,Mp(pj)}; wherein each Rect(m) is the minimum circumscribed rectangle that frames the target object (m), each LabSym(m) is the correct marker of the target object (m), m=1,2,...,Mp(pj); Mp(pj)=the number of targets in the image P(pj); In this embodiment, when pj=1, the corrected annotation file LabPFile(1) is a correct annotation file; after correction, the correct annotation file LabPFile(1)={Rect(m),LabSym(m)|m=1,2,...,9}; wherein each Rect(m) is the minimum enclosing rectangle that encloses the target object (m), each LabSym(m) is the correct marker of the target object (m), m=1,2,...,9; Mp(1)=the number of targets in the image P(1)=9; Step 13.2.4, save the corrected annotation file LabPFile(pj) to the original annotation file LabPFile(pj) in the preprocessing set PreProcSet; In this embodiment, when pj=1, the corrected annotation file LabPFile(1) is saved to the original annotation file LabPFile(1) in the preprocessing set PreProcSet; Step 13.2.5, add 1 to the pj; Step 13.2.6: If pj>PicPn, go to step 13.2.7; otherwise go to step 13.2.2; Step 13.2.7, end the correction process; In this embodiment, after the correction process is completed, PreProcSet={P(pj),LabPFile(pj)|pj=1,2,...,SumJ}; Compression processing; In this embodiment, the specific steps of compression processing are as follows: Step 13.3.1, set the initial value of pj to PStrJ, and set the initial value of cj to CStrJ; wherein pj is the number of the uncompressed picture in the preprocessing set PreProcSet; PStrJ is the starting number of the uncompressed picture in the preprocessing set PreProcSet; cj is the number of the picture in the compressed set; CStrJ is the starting number of the picture in the compressed set; In Example 1, pj=PStrJ=1 is set, and cj=CStrJ=1 is set; Example 2, set pj=PStrJ=56381, set cj=CStrJ=56381; Step 13.3.2, compressing the picture P(pj) in the preprocessing set PreProcSet into a compressed picture CompG(cj); wherein the compressed picture CompG(cj) is a picture with a width of w and a height of h; In this embodiment, when pj=1 and cj=1, the picture P(1) in the preprocessing set PreProcSet is compressed into a compressed picture CompG(1); wherein the compressed picture CompG(1) is a picture with a width of w and a height of h; Step 13.3.3, obtain the annotation file CompGFile(cj) of the compressed image CompG(cj); The steps to obtain the annotation file CompGFile(cj) of the compressed image CompG(cj) are as follows: Step 13.3.3.1. Compress the X-direction boundary value of each minimum bounding rectangle Rect(m) in the annotation file LabPFile(pj) according to the compression ratio w / W(pj) and the Y-direction boundary value according to the compression ratio h / H(pj), and obtain the X-direction boundary value and Y-direction boundary value of the corresponding minimum bounding rectangle CompRect(m) in the annotation file CompGFile(cj), m=1,2,...,Mp(pj); wherein W(pj) and H(pj) are the width and height of the image P(pj) respectively; In this embodiment, when pj=1, cj=1, and m=1, the X-direction boundary value of the minimum bounding rectangle Rect(1) in the annotation file LabPFile(1) is compressed according to the compression ratio w / W(1) and the Y-direction boundary value is compressed according to the compression ratio h / H(1), and the X-direction boundary value and the Y-direction boundary value of the corresponding minimum bounding rectangle CompRect(1) in the annotation file CompGFile(1) are obtained. The compression calculation is as follows: CRsx(1)=int[RSx(1)×w / W(1)], CRex(1)=int[REx(1)×w / W(1)] CRsy(1)=int[RSy(1)×h / H(1)], CRey(1)=int[REy(1)×h / H(1)] Where RSx(1), REx(1), RSy(1), and REy(1) are the left, right, upper, and lower boundary values ​​of Rect(1), respectively; CRsx(1), CRex(1), CRsy(1), and CRey(1) are the left, right, upper, and lower boundary values ​​of CompRect(1), respectively; w=608, h=608, W(1)=1920, H(1)=1080; Similarly, the left boundary value CRsx(m), right boundary value CRex(m), upper boundary value CRsy(m), and lower boundary value CRey(m) of other minimum bounding rectangles CompRect(m) in CompGFile(1) are calculated, where m=2, 3, ..., 9; Step 13.3.3.2. Set each CLabSym(m)=LabSym(m) in CompGFile(cj), m=1,2,...,Mp(pj), and set Mc(cj)=Mp(pj); In this embodiment, when pj=1, cj=1, each CLabSym(m)=LabSym(m) in CompGFile(1) is set to m=1,2,...,Mp(1), and Mc(1)=Mp(1)=9; Step 13.3.3.3, use Mc(cj) and each CompRect(m) and each CLabSym(m), m=1,2,...,Mc(cj), to form the annotation file CompGFile(cj); CompGFile(cj)={CompRect(m),CLabSym(m)|m=1,2,...,Mc(cj)}; In this embodiment, when cj=1, Mc(1) and each CompRect(m) and each LabSym(m), m=1,2,...,9, are used to form a labeling file CompGFile(1); CompGFile(1)={CompRect(m),CLabSym(m)|m=1,2,...,Mc(1)}; Mc(1)=9; CLabSym(1)=CLabSym(2)=12; Step 13.3.4, adding the picture CompG(cj) and the annotation file CompGFile(cj) to the compression set CompSet; In this embodiment, when cj=1, the picture CompG(1) and the annotation file CompGFile(1) are added to the compression set CompSet; Step 13.3.5, add 1 to the pj, and add 1 to the cj; Step 13.3.6: If pj>PicPn, go to step 13.3.7; otherwise go to step 13.3.2; Step 13.3.7, end the compression process; when it ends, CompSet={CompG(cj),CompGFile(cj)|cj=1,2,...,SumJ}; Acquire a processing set of a set target object from the preprocessing set; In this embodiment, the processing set AppointProcSet of the set target object is obtained from the preprocessing set PreProcSet; wherein, AppointProcSet={PImg(pv),LabPIFile(pv)|pv=1,2,...,PV}; PImg(pv) is a picture of the set target object with a width of w and a height of h; LabPIFile(pv) is a labeling file of the picture PImg(pv); PV is the number of pictures in the processing set AppointProcSet of the set target object; Example 1, PV=22570; Example 2, PV=35490; The processing set is composed of the compression set and the processing set of the set target object.

[0063] In this embodiment, the specific steps of using the compression set CompSet and the processing set AppointProcSet of the set target object to form the processing set ProcSet are as follows: Step 13.5.1, set the initial value of aj to AStrJ, set the initial value of cj to CStrJ, set the initial value of pv to PStrV, set AppImgPn to the number of pictures in the processing set of the set target; aj is the number of the picture not added to the processing set; AStrJ is the starting number of the picture not added to the processing set; cj is the number of the picture not added to the processing set in the compression set; CStrJ is the starting number of the picture not added to the processing set in the compression set; pv is the number of the picture not added to the processing set in the processing set AppointProcSet of the set target; PStrV is the starting number of the picture not added to the processing set in the processing set AppointProcSet of the set target; Example 1, set aj=AStrJ=1, set cj=CStrJ=1, set pv=PStrV=1, set AppImgPn=22570; Example 2, set aj=AStrJ=78951, set cj=CStrJ=56381, set pv=PStrV=22571, set AppImgPn=35490; Step 13.5.2, add the picture CompG(cj) in the compression set CompSet to the processing set ProcSet with the name CompP(aj), and add the annotation file CompGFile(cj) in the compression set to the processing set ProcSet with the name CompPFile(aj); Step 13.5.3, add 1 to the aj, and add 1 to the cj; Step 13.5.4: If cj>PicPn, go to step 13.5.5; otherwise go to step 13.5.2; Step 13.5.5, add the picture PImg(pv) in the processing set AppointProcSet of the set target object to the processing set ProcSet with the name CompP(aj), and add the annotation file LabPIFile(pv) in the processing set AppointProcSet of the set target object to the processing set ProcSet with the name CompPFile(aj); Step 13.5.6, add 1 to the aj, and add 1 to the pv; Step 13.5.7: If pv>AppImgPn, go to step 13.5.8; otherwise, go to step 13.5.5; Step 13.5.8, end the processing of forming a processing set using the compression set and the processing set of the set target object.

[0064] After this step is completed, the processing set ProcSet={CompP(aj), CompPFile(aj)|aj=1,2,...,J}; wherein J is the number of pictures in the processing set ProcSet, J=PicPn+AppImgPn; Example 1, J=PicPn+AppImgPn=78950; PicPn=56380, AppImgPn=22570.

[0065] Example 2, J=PicPn+AppImgPn=104760; PicPn=69270, AppImgPn=35490.

[0066] In Example 1 and Example 2, the steps of obtaining the processing set of the set target object from the pre-processing set are as follows: Step 14.1, obtaining an atlas of a set target object from the preprocessing set; In this embodiment, the obtained atlas of the set target object AppointImgSet={APImg(v)|v=1,2,...,V}; wherein APImg(v) is a picture containing the set target object and having a width of w and a height of h; V is the number of pictures in the atlas of the set target object AppointImgSet; Embodiment 1, V=22570; Embodiment 2, V=35490; Step 14.2, using the atlas of the set target object to form a processing set of the set target object; The steps to construct a processing set of a set target object using a set of set target object images are as follows: Step 14.2.1, labeling the atlas of the set target object; In this embodiment, the specific steps of labeling the atlas of the set target object are as follows: Step 14.2.1.1, set the initial value of v to StrV, and set the initial value of pv to PStrV; v is the number of the unlabeled image in the atlas of the set target object; StrV is the starting number of the unlabeled image in the atlas of the set target object; pv is the number of the image in the processing set of the set target object; PStrV is the starting number of the image in the processing set of the set target object; Example 1, set v=StrV=1, set pv=PStrV=1; Example 2, set v=StrV=22571, set pv=PStrV=22571; Step 14.2.1.2, automatically annotate the image APImg(v) using automatic annotation software to obtain the annotation file LabPIFile(pv) of the image APImg(v); In this embodiment, when v=1 and pv=1, the image APImg(1) is automatically annotated using automatic annotation software to obtain the annotation file LabPIFile(1) of the image APImg(1); Step 14.2.1.3, adding the image APImg (v) to the processing set of the set target object with the name PImg (pv), and adding the annotation file LabPIFile (pv) to the processing set of the set target object; In this embodiment, when v=1, pv=1, the picture APImg(1) is added to the processing set AppointProcSet of the set target object with the name PImg(1), and the annotation file LabPIFile(1) is added to the processing set AppointProcSet of the set target object; wherein AppointProcSet={PImg(1),LabPIFile(1)}; Step 14.2.1.4, add 1 to the v, and add 1 to the pv; Step 14.2.1.5, if v>|AppointImgSet|, go to step 14.2.1.6; otherwise go to step 14.2.1.2; Step 14.2.1.6, ending the process of labeling the atlas of the set target object; When this step is completed, AppointProcSet={PImg(pv),LabPIFile(pv)|pv=1,2,...,PV}; wherein PV is the number of images in the processing set AppointProcSet of the set target object; Example 1, PV=22570; Example 2, PV=35490; Step 14.2.2: Correct the processing set of the set target object.

[0067] In this embodiment, the specific steps of correcting the processing set of the set target object are as follows: Step 14.2.2.1, set the initial value of pv to PStrV; pv is the number of the uncorrected annotation file in the processing set of the set target object; PStrV is the starting number of the uncorrected annotation file in the processing set of the set target object; Example 1, set pv=PStrV=1; Example 2: Set pv=PStrV=22571; Step 14.2.2.2, use the modified labeling software LabelImg to judge the image PImg (pv) and the labeling file LabPIFile (pv), if it is judged that the labeling file LabPIFile (pv) is the correct labeling file, go to step 14.2.2.5; otherwise, go to step 14.2.2.3; Step 14.2.2.3, use the label modification software LabelImg to modify the label file LabPIFile (pv) to a correct label file (the definition of the correct label file is similar to step 13.2.3 above); Step 14.2.2.4, saving the corrected annotation file LabPIFile(pv) to the original annotation file LabPIFile(pv) in the processing set AppointProcSet of the set target object; Step 14.2.2.5, add 1 to the pv; Step 14.2.2.6, if pv>|AppointImgSet|, go to step 14.2.2.7; otherwise go to step 14.2.2.2; Step 14.2.2.7: End the process of correcting the processing set of the set target object; when this step is completed, AppointProcSet={PImg(pv),LabPIFile(pv)|pv=1,2,...,PV}.

[0068] In Example 1 and Example 2, the steps of obtaining the atlas of the set target object from the preprocessing set are as follows: Step 15.1, set the initial value of pj to PStrJ, set the initial value of v to StrV, set PreProcPn to the number of images in the preprocessing set; pj is the number of the image that has not been taken in the preprocessing set; PStrJ is the starting number of the image that has not been taken in the preprocessing set; v is the number of the image in the set of the target object; StrV is the starting number of the image in the set of the target object; the image that has not been taken refers to the image of the set target object that has not been obtained; Example 1, set pj=PStrJ=1, set v=StrV=0, set PreProcPn=56380; Example 2, set pj=PStrJ=56381; set v=StrV=56380, set PreProcPn=69270; Step 15.2, obtaining a picture of a set target object from the picture P(pj) and the annotation file LabPFile(pj); the picture P(pj) and the annotation file LabPFile(pj) are both the picture and the annotation file in the preprocessing set; The steps to obtain the image of the set target object from the image P(pj) and the annotation file LabPFile(pj) are as follows: Step 15.2.1, set the initial value of m to 1; wherein m is the number of the target object in the annotation file LabPFile(pj) in the preprocessing set; In this embodiment, when pj=2, the initial value of m is set to 1; Step 15.2.2: If the identifier LabSym(m) of the target object(m) is the identifier of the set target object, go to step 15.2.3; otherwise, go to step 15.2.7; In this embodiment, when pj=2, m=3, the identifier LabSym(3) of the target object (3) is the identifier of the target object 12, and the process goes to step 15.2.3; In this embodiment, when pj=2, m=4, the identifier LabSym(4) of the target object (4) is not the identifier of any set target object, and the process goes to step 15.2.7; In this embodiment, when pj=2, m=5, the identifier LabSym(5) of the target object (5) is the identifier of the green cutting board of the target object 36, and the process goes to step 15.2.3; Step 15.2.3, add 1 to the v; In this embodiment, when pj=2 and m=3, 1 is added to the v; after the addition, v=1; In this embodiment, when pj=2 and m=5, 1 is added to the v; after the addition, v=2; Step 15.2.4, extracting the minimum circumscribed rectangle Rect(m) that frames the target object (m) from the annotation file LabPFile(pj); In this embodiment, when pj=2 and m=3, the minimum circumscribed rectangle Rect(3) enclosing the target object (3) is taken out from the annotation file LabPFile(2); In this embodiment, when pj=2 and m=5, the minimum circumscribed rectangle Rect(5) enclosing the target object (5) is taken out from the annotation file LabPFile(2); Step 15.2.5, according to the minimum circumscribed rectangle Rect(m), obtain the picture APImg(v) from the picture P(pj); wherein the picture APImg(v) is a picture that contains the entire rectangle Rect(m) and has a width of w and a height of h; According to the minimum enclosing rectangle Rect(m), the steps of obtaining the image APImg(v) from the image P(pj) are as follows: Step 15.2.5.1, use formula (4) to calculate the width Rw(m) and height Rh(w) of the minimum circumscribed rectangle Rect(m), Rw(m)=REx(m)-RSx(m)+1 Rh(m)=REy(m)-RSy(m)+1 (4) (4) In the formula, RSx(m), REx(m), RSy(m), and REy(m) are the left boundary value, right boundary value, upper boundary value, and lower boundary value of the rectangle Rect(m), respectively; In this embodiment, when m=3, the width Rw(3) and height Rh(3) of the minimum circumscribed rectangle Rect(3) are calculated using formula (4); In this embodiment, when m=5, the width Rw(5) and height Rh(5) of the minimum circumscribed rectangle Rect(5) are calculated using formula (4); Step 15.2.5.2: If Rw(m)≤w∧Rh(m)≤h, go to step 15.2.5.3; otherwise go to step 15.2.5.4; In this embodiment, when m=3, Rw(3)≤w∧Rh(3)≤h, go to step 15.2.5.3; In this embodiment, when m=5, Rw(5)>w, Rh(5)<h, go to step 15.2.5.4; Step 15.2.5.3, according to the boundary value of Rect(m), obtain the w*h area containing the entire rectangle Rect(m) from the image P(pj), and save the image of this w*h area as the image APImg(v); after this step is completed, go to step 15.2.5.5; wherein, the w*h area is an area with a width of w and a height of h; In this embodiment, when m=3, pj=2, v=1, the w*h area including the entire rectangle Rect(3) is obtained from the image P(2) according to the boundary value of Rect(3), and the image of the w*h area is saved as the image APImg(1); after this step is completed, go to step 15.2.5.5; Step 15.2.5.4, according to the boundary value of Rect(m), obtain the ERw*ERh area containing the entire rectangle Rect(m) from the picture P(pj), compress the image of the ERw*ERh area into a w*h image, and then save the w*h image as the picture APImg(v); wherein the ERw*ERh area is an area with a width of ERw and a height of ERh; ERw and ERh are calculated by formula (5), ERw=Rw(m), ERh=int[h×Rw(m) / w] if Rw(m) / w≥Rh(m) / h ERw=int[w×Rh(m) / h], ERh=Rh(m) else (5) In this embodiment, when m=5, pj=2, and v=2, the ERw*ERh region including the entire rectangle Rect(5) is obtained from the image P(2) according to the boundary value of Rect(5), and the image of the ERw*ERh region is compressed into a w*h image, and then the w*h image is saved as the image APImg(2); wherein the ERw*ERh region is a region with a width of ERw and a height of ERh; ERw and ERh are calculated by formula (5) (because Rw(5) / w>1.0, Rh(5) / h<1.0), so, ERw=Rw(5), ERh=int[h×Rw(5) / w] (5) Step 15.2.5.5, end the process of obtaining the image API mg(v); Step 15.2.6, adding the image APImg(v) to the atlas of the set target object AppointImgSet; In this embodiment, when v=1 and v=2, the picture APImg(1) and the picture APImg(2) are added to the atlas AppointImgSet of the set target object in sequence; Step 15.2.7, add 1 to the m; Step 15.2.8, if m>Mp(pj), go to step 15.2.9, otherwise go to step 15.2.2; where Mp(pj) is the number of targets in the annotation file LabPFile(pj); Step 15.2.9, end the process of obtaining the image of the set target object from the image P(pj) and the annotation file LabPFile(pj); Step 15.3, add 1 to the pj; Step 15.4: If pj>PreProcPn, go to step 15.5; otherwise go to step 15.2; Step 15.5, end the process of obtaining the atlas of the set target object from the preprocessing set.

[0069] In this embodiment, when this step is completed, the atlas of the target object AppointImgSet={APImg(v)|v=1,2,...,V} is set; wherein V is the number of images in the atlas of the set target object AppointImgSet; Example 1, V=22570.

[0070] Example 2, V=35490.

[0071] The effect of the technical solution for generating a neural network is as follows: video images containing various targets are recorded at different time periods and under different environmental conditions, and then pictures containing various targets under different situations and lighting conditions are captured by screenshot software, and all of the pictures are saved in an atlas; when the number of pictures in the atlas is sufficient, each picture in the atlas is automatically annotated to obtain an annotation file of the picture, and the picture and the corresponding annotation file are saved in a preprocessing set; the incorrect annotation file is corrected to a correct annotation file using annotation modification software, and the corrected annotation file is saved in the preprocessing set; and then the corrected annotation file is compressed and processed. Each image in the preprocessing set is compressed into a w*h image, and the annotation file of the image is correspondingly compressed into an annotation file of the compressed image, and then the compressed image and the annotation file of the compressed image are saved in the compression set; on the other hand, the technical solution also obtains a processing set containing set targets (targets containing people, targets containing cutting boards, targets containing knives, and targets containing small animals) from the preprocessing set. This processing set of set targets contains many images of set targets and the annotation files corresponding to each image. The resolution of these images is the resolution of the original images and they are images with a width of w and a height of h; then Each picture and the annotation file of the picture in the compression set and the processing set of the set target object are added to the processing set; after that, the processing set is split into a training set and a test set; the neural network model is then trained with the training set, and the neural network model is tested with the test set; after the accuracy of the neural network model test meets the use requirements, the neural network model is compiled into a neural network for identifying the target object by using a compiler software, and then the neural network for identifying the target object is written into the intelligent analyzer in each monitoring subsystem in the system; the processing set provides the neural network with relatively complete, rich, and extensive training materials and test materials with actual characteristics, so that the neural network has a higher recognition rate and accuracy, and its monitoring reliability, accuracy, and extensiveness are better than the existing technical solutions; on the other hand, the processing set of the set target object is automatically obtained from the preprocessing set by an automatic processing program, which not only greatly reduces the workload of obtaining the picture of the set target object, but also greatly improves the accuracy and consistency of obtaining the picture containing the set target object; the neural network can learn more extensive, richer, more detailed, more complete, and more practical features, thereby improving the generalization ability and robustness of the neural network, so that the neural network has a stronger and more accurate recognition ability and higher processing efficiency.

[0072] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A vision-based catering safety monitoring system, characterized in that: The system includes a control server, N monitoring subsystems, T servers, V NVRs, a GPU server, a system switch, a network, a network device, a network device Wn, S mobile terminals, M clients and a system power supply; the control server is connected to the system switch and connected to the GPU server through the system switch; the control server is connected to the following devices through the system switch, the network and the network device: connected to each of the T servers, connected to each of the V NVRs, connected to the switch in each of the N monitoring subsystems and connected to the intelligent analyzer in the monitoring subsystem through the switch, the control server is connected to each of the S mobile terminals through the system switch and the network, connected to each of the M clients, connected to the network device Wn and connected through the The network device Wn is connected to the external network; the server managing An(t) catering processing places among the T servers is connected to the switches in An(t) monitoring subsystems through the network device and the network, and is connected to the intelligent analyzer in the monitoring subsystem through the switch, t=1,2,...,T, and An(1)+An(2)+...+An(T)=N; the NVR connecting Bn(v) cameras among the V NVRs is connected to at least one switch in the monitoring subsystem through the network device and the network, and is connected to at least one camera in the monitoring subsystem through the switch, and a total of Bn(v) cameras are connected, v=1,2,...,V, and each Bn(v) is a positive integer; the system power supply is connected to the system switch and the network device Wn and provides the required DC power to the system switch and the network device Wn; wherein N, T, V, S and M are all positive integers.

2. The visual-based catering safety monitoring system according to claim 1, characterized in that: The monitoring subsystem includes an intelligent analyzer, a switch, a network device, Q cameras and a power supply; wherein Q is a positive integer; the intelligent analyzer is connected to the switch, and is connected to the network device and each of the Q cameras through the switch, and is connected to the control server through the switch, the network device, the network, and the system switch, and is also connected to one of the T servers through the switch, the network device, and the network; the images collected by the Q cameras in the monitoring subsystem cover the entire area of ​​a catering processing; the power supply is connected to the intelligent analyzer, the switch, the network device, and provides the required DC power for the intelligent analyzer, the switch, and the network device.

3. A method of a visual-based catering safety monitoring system as claimed in claim 1 or claim 2, characterized in that: The steps of the method described are as follows: Step A, generating a neural network; the neural network is a neural network that can identify targets; the targets include at least K targets; the K targets include 28 targets, M1 targets B1, M2 targets B2, and M2 targets B3; K, M1, and M2 are all positive integers; wherein, target 1, target 2, ..., target 28 among the 28 targets are respectively shelves, tables, cabinets, fixed barrels, stoves, range hoods, pillars, mice, cats, weasels, dogs, people, people in standard dress, people who taste food with special tasting utensils, people who wear hats improperly, people who wear masks improperly, people who wear tops improperly, people who wear pants improperly, people who wear shoes improperly, people who smoke cigarettes, people who use mobile phones, people who taste food with a cooking spoon, people who run, people with abnormal limbs, ground without waste, ground with waste The ground, fire, and smoke of the object; M1 target object B1 is the target object of the food basket of M1 color, M2 target object B2 is the target object of the cutting board of M2 color, and M2 target object B3 is the target object of the knife handle of M2 color; the people are not non-compliant people, and non-compliant people include people who wear hats irregularly, people who wear masks irregularly, people who wear tops irregularly, people who wear pants irregularly, people who wear shoes irregularly, people who smoke cigarettes, people who use mobile phones, people who taste food with cooking spoons, people who run, and people with abnormal limbs; the waste is waste with width ≥ ThingW and height ≥ ThingH; the fire is fire with width ≥ ThingW and height ≥ ThingH; the ThingW and ThingH correspond to the actual width in the X direction and the actual height in the Y direction of the image captured by the camera, respectively; Step B, monitoring and processing; The steps of monitoring and processing are as follows: Step B.1, set the initial value of q to 1; q is the number of the monitoring area in the monitoring subsystem; Step B.2: If the execution time of this step is within the monitoring time of the camera in the monitoring area q, go to step B.3; otherwise, go to step B.7; Step B.3, identify the target object based on the neural network and obtain the identification set IdenSet(q); Step B.4, monitoring events according to the identification set IdenSet(q) or monitoring events according to the difference graph; Step B.5: If an event is detected, go to step B.6; otherwise go to step B.7; Step B.6: event processing; Step B.7, adding 1 to the q; Step B.8: If q>Q, go to step B.1; otherwise go to step B.2; wherein Q is the number of monitoring areas in the monitoring subsystem.

4. The method of the vision-based catering safety monitoring system according to claim 3, characterized in that: The steps of identifying the target object based on the neural network and obtaining the identification set IdenSet(q) are as follows: Step 4.1, the intelligent analyzer captures a frame of image from the video image output by the camera of the monitoring area q; the width of the image is W(q) and the height is H(q); Step 4.2, compressing the image into a compressed image; the compressed image is an image with a width of w and a height of h; wherein w and h are the width and height of the image processed by the neural network, and w and h need to satisfy the following relationship: w≥DetectW×IdnXPn / ThingW h≥DetectH×IdnYPn / ThingH Wherein, DetectW and DetectH are the width and height of the camera monitoring area respectively; IdnXPn and IdnYPn are the minimum pixel point in the X direction and the minimum pixel point in the Y direction of the object recognized by the neural network respectively; Step 4.3, identifying the target object from the compressed image based on the neural network, and obtaining the identification set IdenSet(q); the elements in the identification set IdenSet(q) at least include the number of background targets identified from the compressed image, the minimum circumscribed rectangle that frames each target object, and the identifier of each target object; the identifier of target object k is the marker of target object k identified by the neural network, and the marker of target object k is a symbol assigned to target object k that is different from other targets, k=1,2,...,K; wherein k is the number of the target object; K is the number of the target objects; Step 4.4: End the process of identifying the target object based on the neural network and obtain the identification set IdenSet(q).

5. The method of the vision-based catering safety monitoring system according to claim 3, characterized in that: The steps of monitoring events according to the identification set IdenSet(q) or monitoring events according to the difference graph are as follows: According to the identification set IdenSet(q), monitoring smoke events; According to the identification set IdenSet(q), the steps of monitoring smoke events are as follows: According to the identification set IdenSet(q), the maximum number of background objects is recorded; the background objects include shelves, tables, cabinets, fixed barrels, stoves, range hoods and pillars; the number of background objects is the number of background objects in the identification set IdenSet(q); Monitoring smoke events according to the identification set IdenSet(q) and the maximum number of background targets; The event of detecting smoke is when the number of background targets in the identification set IdenSet(q) is less than the maximum number of background targets; According to the identification set IdenSet(q), a fire event is monitored; the fire event is detected when at least one target object identifier in the identification set IdenSet(q) is an identifier of the target object being fire, and the target object is not within the range of any stove; According to the identification set IdenSet(q), monitoring the event of the ground with waste; The event of detecting a ground with waste means that in the identification set IdenSet(q), at least one identifier of a target object is an identifier of a ground with waste; According to the identification set IdenSet(q), an event of incorrect food basket position is detected; the event of incorrect food basket position is detected when at least one target object identifier in the identification set IdenSet(q) is an identifier of the target object being the food basket, and the target object is not within the range box of the target object; the food basket is a food basket of M1 colors; If the execution time of this step is in the time period of monitoring small animals, then the event of monitoring the presence of small animals is detected according to the difference map; the event of detecting the presence of small animals means that at least one target object identifier in the small animal identification symbol set IdenAnimalSymSet(q) is an identifier of the target object being a small animal; the small animals include mice, cats, weasels, and dogs; According to the identification set IdenSet(q), an identification symbol set IdenPeopleSymSet(q) and a blade position identifier BdAndKnPlace(q) are obtained; wherein the elements in the identification symbol set IdenPeopleSymSet(q) are identifiers of target objects including people; According to the identification symbol set IdenPeopleSymSet(q), the event of a person who does not meet the specifications is monitored; the event of a person who does not meet the specifications is detected when at least one of the identifiers of the target object in the identification symbol set IdenPeopleSymSet(q) is an identifier of a person who does not meet the specifications, or the ASM condition is also satisfied; the target objects that do not meet the specifications include target 15, target 16, ..., target 24; the ASM condition is that if the identifier of the target object is an identifier of a continuous action target object, the event of a person who does not meet the specifications is detected when ActN / s≥SetMinN / s∧ ActTime≥MinTime; wherein, 2 times / s≤SetMinN / s≤5 times / s, 2s≤MinTime≤4s; ActN / s and SetMinN / s are respectively the number of times the identifier of the continuous motion target is detected within one second and the set minimum number of times the identifier of the continuous motion target is detected within one second; ActTime and MinTime are respectively the duration of the detected continuous motion target identifier and the set minimum duration of the detected continuous motion target identifier; the continuous motion targets are targets 23 and 24; According to the knife board position identifier BdAndKnPlace(q), the event of incorrect cutting board position or incorrect cutting tool position is monitored; the event of incorrect cutting board position or incorrect cutting tool position is detected when the knife board position identifier BdAndKnPlace(q)=false; the cutting board includes cutting boards of M2 colors; the cutting tools include cutting tools with handles of M2 colors.

6. The method of the vision-based catering safety monitoring system according to claim 3 or claim 5, characterized in that: The steps for setting the monitoring time of the camera in monitoring area q and the time period for monitoring small animals and the range frame of monitoring area q are as follows: For each camera in each monitoring area q, the monitoring start time and monitoring end time of each monitoring time period c in a day are set, and the time period for monitoring small animals is set, c=1,2,...,C(q), q=1,2,...,Q; where c is the number of the monitoring time period formed in the order of the monitoring time in a day; C(q) is the number of monitoring time periods in a day, and C(q) is a positive integer; The setting of the range frame of the monitoring area q refers to setting the range frame of the target object with a limited range in the camera acquisition image of each monitoring area q, q=1,2,...,Q; the range frame corresponds to the range of the actual stove, or corresponds to the correct range where the target object is actually placed; the targets with limited ranges include target 5 stove, target M1 color food basket, target M2 color chopping board, and target M2 color knife handle.

7. The method of the vision-based catering safety monitoring system according to claim 5, characterized in that: The steps of monitoring the event of small animals according to the difference map are as follows: Step 7.1, set the small animal identification symbol set IdenAnimalSymSet(q) to an empty set; where q is the number of the monitoring area; Step 7.2, obtaining a difference image from the video image; the difference image is a difference image obtained by subtracting the current frame image output by the camera of the monitoring area q from the previous L frames of image output; wherein 1≤L≤12; Step 7.3, obtaining a difference rectangle set DiffRectSet from the difference image; Step 7.4: If |DiffRectSet|≥1, go to step 7.5; otherwise, go to step 7.6; Step 7.5, identify and process the difference rectangle set DiffRectSet to obtain the small animal identification symbol set IdenAnimalSymSet(q); Step 7.6, end the processing of the event of monitoring the presence of small animals based on the difference map.

8. The method of the vision-based catering safety monitoring system according to claim 5, characterized in that: The steps of obtaining the identification symbol set IdenPeopleSymSet(q) and the blade position identifier BdAndKnPlace(q) according to the identification set IdenSet(q) are as follows: Step 8.1, set the initial value of n to 1, the initial value of n1 to 0, set the identification symbol set IdenPeopleSymSet(q) to an empty set, and set the blade position identifier BdAndKnPlace(q)=true; wherein n is the number of the target object in the identification set IdenSet(q); n1 is the number of the target object containing people in the identification set IdenSet(q); the targets containing people include target 12, target 13, ..., target 24; Step 8.2: If the identifier of the target object (n) is the identifier of a target object including a person, or the identifier of a target object including a chopping board, or the identifier of a target object including a knife, then go to step 8.3; otherwise, go to step 8.9; the target object including a chopping board includes targets of chopping boards of M2 colors; the target object including a knife includes targets of knives with handles of M2 colors; Step 8.3, obtaining an image ImgP(n) from the image according to the rectangle CompPRect(q,n); the image ImgP(n) is an image with a width of w and a height of h; the rectangle CompPRect(q,n) is the minimum circumscribed rectangle that frames the target object (n) in the identification set IdenSet(q); Step 8.4, using the neural network to identify the image ImgP(n), and obtaining the identifier IdenSym(q,n) of the image ImgP(n); Step 8.5: If the identifier of the target object (n) is an identifier of a target object including a person, go to step 8.6; otherwise, go to step 8.8; Step 8.6, add 1 to the n1; Step 8.

7. Add the identifier IdenSym(q,n) to the identifier set with the name IdenPeopleSym(q,n1) IdenPeopleSymSet(q); After this step is completed, go to step 8.9; Step 8.8, obtain the knife plate position identifier BdAndKnPlace(q); The steps to obtain the blade position identifier BdAndKnPlace(q) are as follows: Step 8.8.1: If the target object corresponding to the identifier IdenSym(q,n) is not within the range box of the target object, go to step 8.8.2; otherwise, go to step 8.8.3; Step 8.8.2, set the knife position identifier BdAndKnPlace(q)=false; Step 8.8.3, end the process of obtaining the blade position identifier BdAndKnPlace(q); Step 8.9, add 1 to the n; Step 8.10: If n>Nc(q), go to step 8.11; otherwise go to step 8.2; wherein Nc(q) is the number of targets in the identification set IdenSet(q); Step 8.11, end the processing of obtaining the identification symbol set IdenPeopleSymSet(q) and the blade position identifier BdAndKnPlace(q) according to the identification set IdenSet(q).

9. The method of the vision-based catering safety monitoring system according to claim 8, characterized in that: The steps of obtaining the image ImgP(n) from the image according to the rectangle CompPRect(q,n) are as follows: Step 9.1, using formula (1) to calculate the left boundary value PSx(n), the right boundary value PEx(n), the upper boundary value PSy(n), and the lower boundary value PEy(n) of the rectangle CompPRect(q,n) in the rectangle PREct(n) of the image, PSx(n)=int[CPsx(n)×W(q) / w], PEx(n)=int[CPex(n)×W(q) / w] PSy(n)=int[CPsy(n)×H(q) / h], PEy(n)=int[CPey(n)×H(q) / h] (1) (1) In the formula, CPsx(n), CPex(n), CPsy(n), and CPey(n) are the left boundary value, right boundary value, upper boundary value, and lower boundary value of the rectangle CompPRect(q,n), respectively; W(q) and H(q) are the width and height of the image, respectively; int[] is an integer operation; Step 9.2, use formula (2) to calculate the width Pw(n) and height Ph(n) of the rectangle PREct(n), Pw(n)=PEx(n)-PSx(n)+1 Ph(n)=PEy(n)-PSy(n)+1 (2) Step 9.3: If Pw(n)≤w∧Ph(n)≤h, go to step 9.4; Otherwise, go to step 9.5; Step 9.4, according to the boundary value of PREct(n), obtain the image ImgP(n) containing the entire rectangle PREct(n) from the image; after this step is completed, go to step 9.7; the image ImgP(n) is an image with a width of w and a height of h; Step 9.5, according to the boundary value of PREct(n), obtain the image EImgP(n) containing the entire rectangle PREct(n) from the image; the width of EImgP(n) is EPw(n) and the height is EpH(n); EPw(n) and EpH(n) are calculated by formula (3), EPw(n)=Pw(n),EPh(n)=int[h×Pw(n) / w] if Pw(n) / w≥Ph(n) / h EPw(n)=int[w×Ph(n) / h],EPh(n)=Ph(n) else (3) Step 9.6, compress the image EImgP(n) into an image ImgP(n) with a width of w and a height of h; Step 9.7, end the process of obtaining the image ImgP(n) from the image according to the rectangle CompPRect(q,n).

10. The method of the vision-based catering safety monitoring system according to claim 3, characterized in that: The steps of event processing are as follows: Generate event records; the generating of event records refers to the intelligent analyzer generating an event record set; the event record set includes at least the following elements: event name, time of event occurrence, location of event occurrence, event type, picture reflecting the event, or also includes GIF animation, or also includes short video, or also includes video recording reflecting the event; Uploading event records; the uploading event records refers to the intelligent analyzer uploading the event record set to the control server through the switch, network equipment, network, and system switch; Recording events: recording events means that the control server saves the event record set to the storage device of the control server; Event record publishing; event record publishing means that the control server publishes the event record set to the client with the permission to view the event record set and the mobile phone with the permission to view the event record set through the system switch and the network; Event alarm processing; the event alarm processing means that if the event is judged to be a smoke event, an alarm notification is made; if the event is judged to be a fire event, an alarm notification and event transmission are made; the alarm notification means that the control server sends the event information of smoke or fire to the relevant mobile phones and clients through the system switch and network; the event transmission means that the control server transmits the event record set to the system set place in the external network through the system switch, network, and network device Wn.

11. The method of the vision-based catering safety monitoring system according to claim 3, characterized in that: The steps of generating a neural network are as follows: Step 11.1, set the number of acquired pictures PicNum, and set the following sets as empty sets: atlas, preprocessing set, compression set, atlas of set target objects, processing set of set target objects, processing set; wherein PicNum is a positive integer ≥ 3000; the set targets include at least target 8, target 9, ..., target 26, targets of cutting boards of M2 colors, and targets of knives with knife handles of M2 colors; Step 11.2, obtain the atlas; Step 11.3, obtaining a processing set using the atlas; Step 11.4, splitting the processing set into a training set and a test set; Step 11.5, training the neural network model; training the neural network model means training the neural network model with each set of images and files in the training set; a set of images and files means a picture and a labeling file of the picture; Step 11.6, testing the neural network model; testing the neural network model means testing the neural network model with each set of images and files in the test set, and counting the number of each target object tested and the number of each target object tested correctly; Step 11.7, count the test accuracy of each target object k, k=1,2,...,K; Step 11.8: If the test accuracy of each target object k is ≥ the set accuracy, go to step 11.10; otherwise, go to step 11.9; Step 11.9, increase the PicNum value, and then go to step 11.2; increasing the PicNum value means that the PicNum after increase = PicNum before increase × ChgScaleV, where 1.1≤ChgScaleV≤2.4; Step 11.10, compiling the neural network model into a neural network; the neural network is a neural network that can identify the target object; Step 11.11, writing the neural network into each intelligent analyzer in the system; Step 11.12, end the process of generating the neural network.

12. The method of the vision-based catering safety monitoring system according to claim 11, characterized in that: The steps of obtaining the atlas are as follows: Obtaining video; obtaining video means obtaining D1 video clips containing the target object; D1 is a positive integer; Get pictures; getting pictures means capturing pictures from each video recording d1. d1=StrDx,StrDx+1,...,D1, and add the captured image to the temporary storage set TmpSet(d), d=StrD,StrD+1,...,D; where d1 is the number of the video recording segment without the captured image; StrDx is the starting number of the video recording segment without the captured image; d is the number of the temporary storage set; StrD is the starting number of the temporary storage set; D is the number of the temporary storage set, and D is a positive integer; Filter the images; filtering the images means filtering each temporary set TmpSet(d) that has not been filtered, d=StrD,StrD+1,...,D; filtering the temporary set TmpSet(d) means saving the different images in the temporary set TmpSet(d) into the atlas; two images in the temporary set TmpSet(d) are different if at least one of the targets in the two images is different, or the number of targets in the two images is different, or there is at least one target k such that PArea(d,a1,k)∩PArea(d,a2,k) is an empty set, a1≠a2∧k∈Z[1,K ]; wherein PArea(d,a1,k) and PArea(d,a2,k) are respectively the area where the target object k in the image Pt(d,a1) is located and the area where the same target object k in the image Pt(d,a2) is located, and the operation of PArea(d,a1,k)∩PArea(d,a2,k) is an operation performed on the image Pt(d,a1) and the image Pt(d,a2) in the same image coordinate system; wherein the image Pt(d,a1) and the image Pt(d,a2) are both images in the temporary set TmpSet(d); Z[m1,m2] is a set from integers m1 to integers m2, m1≤m2.

13. The method of the vision-based catering safety monitoring system according to claim 11, characterized in that: The steps of obtaining the processing set using the atlas are as follows: Annotation processing; Annotation processing refers to automatically annotating each unannotated picture in the atlas using automatic annotation software to obtain an annotation file of the picture, and then adding the picture and the annotation file of the picture to the preprocessing set; the annotation file at least includes the following data of each target object identified from the annotated picture: a rectangle marking the position of the target object, and a marker of the target object; wherein the marker of the target object is a different symbol assigned to different targets; Correction processing; Correction processing refers to correcting the uncorrected annotation files in the preprocessing set by using modification annotation software, and saving the corrected annotation files to the original annotation files in the preprocessing set; Correcting the annotation files means that if there are incorrect annotations in the annotation files, the annotation files are corrected to be correct annotation files; the correct annotation files at least include the following data of each target object in the annotated image: the minimum circumscribed rectangle that frames the target object, and the correct marker of the target object; Compression processing; compression processing refers to compressing each uncompressed image in the preprocessing set to obtain a compressed image of the image, and compressing the annotation file of the image to obtain an annotation file of the compressed image; then adding the compressed image and the annotation file of the compressed image to the compression set; the compressed image is an image with a width of w and a height of h; the annotation file of the compressed image is an annotation file that annotates the compressed image; Acquire a processing set of a set target object from the preprocessing set; The processing set is composed of the compression set and the processing set of the set target object.

14. The method of the vision-based catering safety monitoring system according to claim 13, characterized in that: The steps of obtaining the processing set of the set target object from the preprocessing set are as follows: Step 14.1, obtaining an atlas of a set target object from the preprocessing set; Step 14.2, using the atlas of the set target object to form a processing set of the set target object; The steps to construct a processing set of a set target object using a set of set target object images are as follows: Step 14.2.1, annotating the atlas of the set target object; annotating the atlas of the set target object means automatically annotating each unannotated picture in the atlas of the set target object using automatic annotation software, obtaining an annotation file of the picture, and adding the picture and the annotation file of the picture to the processing set of the set target object; Step 14.2.2, modifying the processing set of the set target object; Correcting the processing set of the set target object means correcting each uncorrected annotation file in the processing set of the set target object using modification annotation software, and saving the correctly corrected annotation file to the original annotation file of the processing set of the set target object.

15. The method of the vision-based catering safety monitoring system according to claim 14, characterized in that: The steps of obtaining the atlas of the set target object from the preprocessing set are as follows: Step 15.1, set the initial value of pj to PStrJ, set the initial value of v to StrV, set PreProcPn to the number of images in the preprocessing set; pj is the number of the image that has not been taken in the preprocessing set; PStrJ is the starting number of the image that has not been taken in the preprocessing set; v is the number of the image in the set of the target object; StrV is the starting number of the image in the set of the target object; the image that has not been taken refers to the image of the set target object that has not been obtained; Step 15.2, obtaining a picture of a set target object from the picture P(pj) and the annotation file LabPFile(pj); the picture P(pj) and the annotation file LabPFile(pj) are both pictures and annotation files in the preprocessing set; The steps to obtain the image of the set target object from the image P(pj) and the annotation file LabPFile(pj) are as follows: Step 15.2.1, set the initial value of m to 1; wherein m is the number of the target object in the annotation file LabPFile(pj) in the preprocessing set; Step 15.2.2: If the identifier of the target object (m) is the identifier of the set target object, go to step 15.2.3; otherwise, go to step 15.2.7; Step 15.2.3, add 1 to the v; Step 15.2.4, extracting the minimum circumscribed rectangle Rect(m) that frames the target object (m) from the annotation file LabPFile(pj); Step 15.2.5, according to the minimum circumscribed rectangle Rect(m), obtain the picture APImg(v) from the picture P(pj); wherein the picture APImg(v) is a picture that contains the entire rectangle Rect(m) and has a width of w and a height of h; According to the minimum enclosing rectangle Rect(m), the steps of obtaining the image APImg(v) from the image P(pj) are as follows: Step 15.2.5.1, use formula (4) to calculate the width Rw(m) and height Rh(w) of the minimum circumscribed rectangle Rect(m), Rw(m)=REx(m)-RSx(m)+1 Rh(m)=REy(m)-RSy(m)+1 (4) (4) In the formula, RSx(m), REx(m), RSy(m), and REy(m) are the left boundary value, right boundary value, upper boundary value, and lower boundary value of the rectangle Rect(m), respectively; Step 15.2.5.2: If Rw(m)≤w∧Rh(m)≤h, go to step 15.2.5.3; otherwise go to step 15.2.5.4; Step 15.2.5.3, according to the boundary value of Rect(m), obtain the w*h area containing the entire rectangle Rect(m) from the image P(pj), and save the image of this w*h area as the image APImg(v); after this step is completed, go to step 15.2.5.5; wherein, the w*h area is an area with a width of w and a height of h; Step 15.2.5.4, according to the boundary value of Rect(m), obtain the ERw*ERh area containing the entire rectangle Rect(m) from the picture P(pj), compress the image of the ERw*ERh area into a w*h image, and then save the w*h image as the picture APImg(v); wherein the ERw*ERh area is an area with a width of ERw and a height of ERh; ERw and ERh are calculated by formula (5), ERw=Rw(m), ERh=int[h×Rw(m) / w] if Rw(m) / w≥Rh(m) / h ERw=int[w×Rh(m) / h], ERh=Rh(m) else (5) Step 15.2.5.5, end the process of obtaining the image API mg(v); Step 15.2.6, adding the image APImg(v) to the atlas of the set target object; Step 15.2.7, add 1 to the m; Step 15.2.8, if m>Mp(pj), go to step 15.2.9, otherwise go to step 15.2.2; where Mp(pj) is the number of targets in the annotation file LabPFile(pj); Step 15.2.9, end the process of obtaining the image of the set target object from the image P(pj) and the annotation file LabPFile(pj); Step 15.3, add 1 to the pj; Step 15.4: If pj>PreProcPn, go to step 15.5; otherwise go to step 15.2; Step 15.5, end the process of acquiring the atlas of the set target object.

Images