Model updating method for high-altitude parabola, high-altitude parabola detection system and storage medium

Abstract

The application relates to the field of high-altitude projectile detection, and provides a high-altitude projectile model updating method, which comprises the following steps: acquiring collected monitoring video data, running a first model to process the monitoring video data, and obtaining a first recognition result; when it is determined that the first recognition result is a high-altitude projectile event, running a second model to process the monitoring video data, and obtaining a second recognition result; determining a target updating strategy for the first model and / or the second model according to the second recognition result; and iteratively updating the first model and / or the second model according to the target updating strategy. When the recognition result of the first model is a high-altitude projectile event, the target updating strategy of the detection model is determined according to the recognition result of the second model, the detection model is updated according to the target updating strategy, and a new detection model is obtained, so that the recognition accuracy of the detection model used for recognizing high-altitude projectiles can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of high-altitude litter detection, and in particular to a high-altitude litter model updating method, a high-altitude litter detection system and a storage medium. BACKGROUND

[0002] High-altitude litter not only is an uncivilized behavior, but also threatens people and property on the ground. In order to effectively reduce or even avoid high-altitude litter, the state has clearly defined the crime of high-altitude litter, but the places where high-altitude litter events occur are mostly high-altitude floors, there are few witnesses, and the littering time is short, making it difficult for law enforcement departments to hold the litterers accountable. Therefore, it is extremely important to realize intelligent identification of high-altitude litter.

[0003] Currently, image data is mainly collected through a monitoring device, and the collected image data is input into a high-altitude litter detection model to detect whether a high-altitude litter event occurs. Since the recognition accuracy of the high-altitude litter detection model for high-altitude litter depends on the comprehensiveness of the training sample data, and the training sample data of high-altitude litter events is difficult to collect comprehensively, how to improve the recognition accuracy of the high-altitude litter detection model for high-altitude litter is a problem to be solved at present. SUMMARY

[0004] The embodiments of the present application provide a high-altitude litter model updating method, a high-altitude litter detection system and a storage medium, which help to improve the recognition accuracy of the high-altitude litter detection model for high-altitude litter.

[0005] In a first aspect, the embodiments of the present application provide a high-altitude litter model updating method applied to a high-altitude litter detection system, wherein the high-altitude litter detection system comprises a first model and a second model for identifying a high-altitude litter event, and the method comprises:

[0006] acquiring collected monitoring video data and running the first model to process the monitoring video data to obtain a first recognition result;

[0007] when it is determined that the first recognition result is a high-altitude litter event, running the second model to process the monitoring video data to obtain a second recognition result;

[0008] determining a target updating strategy for the first model and / or the second model according to the second recognition result;

[0009] iteratively updating the first model and / or the second model according to the target updating strategy.

[0010] In a second aspect, the embodiments of the present application also provide a high-altitude littering detection system, the high-altitude littering detection system comprising a first device and a second device, the first device being in communication connection with the second device, the first device comprising a first model for identifying a high-altitude littering event, the second device comprising a second model for identifying a high-altitude littering event, wherein:

[0011] The first device is configured to acquire monitoring video data, run the first model to process the monitoring video data, and obtain a first identification result.

[0012] The first device is further configured to, when it is determined that the first identification result is a high-altitude littering event, send the monitoring video data to the second device.

[0013] The second device is configured to, after receiving the monitoring video data, run the second model to process the monitoring video data, and obtain a second identification result.

[0014] The second device is further configured to determine a target update strategy for the first model and / or the second model according to the second identification result.

[0015] The second device is further configured to iteratively update the first model and / or the second model according to the target update strategy.

[0016] In a third aspect, the embodiments of the present application also provide a storage medium for computer-readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the steps of any of the model updating methods for high-altitude littering provided in the specification of the present application.

[0017] The model updating method for high-altitude littering provided by the embodiments of the present application can adaptively determine a target update strategy for the first model and / or the second model according to the second identification result, and adaptively update the first model and / or the second model according to the target update strategy, so as to obtain a new first model and / or a new second model, thereby enabling the new detection model to adapt to more scenarios of high-altitude littering identification, and greatly improving the identification accuracy of the high-altitude littering detection model. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0019] Figure 1 is a scene schematic diagram of the model updating method of the high-altitude projectile provided by the embodiments of the present application;

[0020] Figure 2 is a step flow schematic diagram of the model updating method of the high-altitude projectile provided by the embodiments of the present application;

[0021] Figure 3 is Figure 2 is a sub-step flow schematic diagram of the model updating method of the high-altitude projectile in

[0022] Figure 4 is an iterative updating process schematic diagram of the first model in the embodiments of the present application;

[0023] Figure 5 is a schematic diagram of the cumulative frame difference graph provided by the embodiments of the present application;

[0024] Figure 6 is a schematic diagram of the current video frame provided by the embodiments of the present application;

[0025] Figure 7 is a schematic diagram of the synthesized image provided by the embodiments of the present application;

[0026] Figure 8 is a schematic diagram of the image obtained by processing the synthesized image by the first model provided by the embodiments of the present application;

[0027] Figure 9 is a step flow schematic diagram of another model updating method of the high-altitude projectile provided by the embodiments of the present application;

[0028] Figure 10 is a structural schematic block diagram of a high-altitude projectile detection system provided by the embodiments of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0030] The flowcharts shown in the drawings are merely illustrative, and are not necessarily required to include all the contents and operations / steps, nor necessarily executed in the order described. For example, some operations / steps can also be decomposed, combined or partially merged, so the actual execution order can be changed according to actual conditions.

[0031] It should be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms "a", "an" and "the" are intended to include the plural forms as well.

[0032] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0033] High-altitude throwing is not only an uncivilized behavior, but also a threat to people and property on the ground. In order to effectively reduce or even avoid high-altitude throwing, the state has clearly defined the crime of high-altitude throwing. However, the high-altitude throwing event usually occurs on high floors, and there are few witnesses, and the throwing time is short, making it difficult for law enforcement departments to hold the thrower accountable. Therefore, it is extremely important to realize intelligent identification of high-altitude throwing. At present, image data is mainly collected through a monitoring device, and then the collected image data is input into a high-altitude throwing detection model to detect whether a high-altitude throwing event has occurred. Since the recognition accuracy of the high-altitude throwing detection model for high-altitude throwing depends on the comprehensiveness of the training sample data, and the training sample data of the high-altitude throwing event is difficult to collect comprehensively, how to improve the recognition accuracy of the high-altitude throwing detection model for high-altitude throwing is a problem to be solved at present.

[0034] To solve the above problems, the embodiment of the present application provides a model updating method for high-altitude throwing, a high-altitude throwing detection system and a storage medium. The method obtains collected monitoring video data, and runs a first model to process the monitoring video data to obtain a first recognition result. When the first recognition result of the first model is a high-altitude throwing event, a second model is run to process the monitoring video data to obtain a second recognition result. According to the second recognition result, a target updating strategy for the first model and / or the second model can be adaptively determined, and the first model and / or the second model can be adaptively updated according to the target updating strategy to obtain a new first model and / or a new second model. Thus, the new detection model can adapt to more scene high-altitude throwing recognition, greatly improving the recognition accuracy of the high-altitude throwing detection model.

[0035] Please refer to Figure 1 , Figure 1 is a scene schematic diagram for implementing the model updating method for high-altitude throwing provided by the embodiment of the present application. As shown inFigure 1 As shown, the scenario includes one or more monitoring devices 100, a first device 200, and a second device 300, the monitoring device 100 is communicatively connected with the first device 200, the first device 200 is communicatively connected with the second device 300, the first device 200 includes a first model for identifying a high-altitude littering event, and the second device 300 includes a second model for identifying a high-altitude littering event.

[0036] The one or more monitoring devices 100 can be connected with the first device 200 through wireless or wired connection. The monitoring device 100 can include an image acquisition device, for example, the image acquisition device is a high-definition camera, an infrared camera, a fisheye camera, or a panoramic camera, etc. The one or more monitoring devices 100 can be deployed at different positions of a building park, a residential area, etc. to collect monitoring video data at different positions of the building park, the residential area, etc. and send the collected monitoring video data to the first device 200. The first device 200 runs the first model to process the monitoring video data to obtain an identification result of a high-altitude littering event, so as to identify the high-altitude littering event and ensure the safety of residents or vehicles in the building park, the residential area, etc. It should be understood that the orientations of the one or more monitoring devices 100 can be the same or different, which is not limited in the embodiment.

[0037] The first device 200 can include a notebook computer, a personal computer (PC), an edge detection device, a server, etc. The second device 300 can include a notebook computer, a personal computer (PC), an edge detection device, a server, etc. The server can be a stand-alone server, a server cluster composed of multiple servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and big data and artificial intelligence platforms, etc. basic cloud computing services. For example, the first device 200 is a personal computer, and the second device 300 is a server. For another example, the first device 200 and the second device 300 are both servers.

[0038] In an embodiment, the first device 200 can be communicatively connected with one or more display devices, and the one or more display devices are used to display monitoring video data collected by the one or more monitoring devices 100. It should be noted that the display device includes a smart display screen, a smart phone, a tablet computer, a personal computer, a notebook computer, or other electronic devices with a display screen, and the embodiment of the present application does not make specific limitation thereon. The display device is provided with a display screen, and the display screen includes an LED display screen, an OLED display screen, an LCD display screen, etc.

[0039] In an embodiment, the first device 200 obtains the monitoring video data collected by the monitoring device 100, processes the monitoring video data by running the first model to obtain a first recognition result, and sends the monitoring video data to the second device 300 when it is determined that the first recognition result is a high-altitude littering event; the second device 300 processes the monitoring video data by running the second model to obtain a second recognition result; the second device 300 determines a target update strategy for the first model and / or the second model according to the second recognition result; and iteratively updates the first model and / or the second model according to the target update strategy. When the second recognition result is a high-altitude littering event, the second device 300 sends a high-altitude littering prompt information to the first device 200, and the first device 200 outputs the high-altitude littering prompt information. For example, the first device 200 outputs the high-altitude littering prompt information to a display device for display, so that a user can check whether a high-altitude littering event really occurs in the area where the high-altitude littering event is recognized in time.

[0040] In the following, the model update method for high-altitude littering provided by the embodiments of the present application will be described in detail in combination with the scenario in Figure 1 It should be noted that, Figure 1 The scenario in

[0041] Please refer to Figure 2 , Figure 2 is a step flowchart of a model update method for high-altitude littering provided by an embodiment of the present application. The model update method for high-altitude littering is applied to a high-altitude littering detection system, and the high-altitude littering detection system includes a first model and a second model for recognizing a high-altitude littering event.

[0042] As shown in Figure 2 , the model update method for high-altitude littering can include steps S101 to S104.

[0043] Step S101, obtaining collected monitoring video data, and processing the monitoring video data by running the first model to obtain a first recognition result.

[0044] The first model can be a detection model, and the second model can be a classification model. The network structure complexity of the second model is greater than that of the first model. The high-altitude littering detection system is in communication connection with one or more monitoring devices. The high-altitude littering detection system can include a first device. The first model and the second model can be deployed on the first device. The high-altitude littering detection system can include a first device and a second device. The first device is in communication connection with the second device. The first device includes the first model, and the second device includes the second model. The first device is in communication connection with one or more monitoring devices.

[0045] In an embodiment, the first model is a pre-trained first neural network model, and the second model is a pre-trained second neural network model, and the first neural network model is different from the second neural network model.

[0046] The first neural network model can be trained in the following manner: obtaining first sample data, wherein the first sample data includes a plurality of first positive samples and a plurality of first negative samples, the first positive samples include high-altitude projectile image and first labels, and the first negative samples include background images and second labels; iteratively training the first neural network model according to the first sample data until the first neural network model converges, thereby obtaining the first model. The first label is used to indicate that the classification result of the high-altitude projectile image is a high-altitude projectile event, i.e., a high-altitude projectile occurs, and the second label is used to indicate that the classification result of the background image is a non-high-altitude projectile event, i.e., no high-altitude projectile occurs.

[0047] The second neural network model can be trained in the following manner: obtaining second sample data, wherein the second sample data includes a plurality of second positive samples and a plurality of second negative samples, the second positive samples include high-altitude projectile image and first labels, and the second negative samples include background images and second labels; iteratively training the second neural network model according to the second sample data until the second neural network model converges, thereby obtaining the second model. The positive and negative sample ratios of the first sample data and the second sample data are different, for example, the positive and negative sample ratio of the first sample data is 1:3, and the positive and negative sample ratio of the second sample data is 1:1.

[0048] In an embodiment, the first model is a pre-trained first neural network model, and the second model is a pre-trained second neural network model, and the first neural network model is different from the second neural network model.

[0049] Step S102, when it is determined that the first recognition result is a high-altitude projectile event, running the second model to process the monitoring video data to obtain a second recognition result.

[0050] In an embodiment, when the first recognition result is determined to be the high-altitude littering event, the first device sends the monitoring video data to the second device; when the second device receives the monitoring video data sent by the first device, the second device runs the second model to process the monitoring video data to obtain a second recognition result. The second recognition result includes the high-altitude littering event and the non-high-altitude littering event.

[0051] In an embodiment, when the second recognition result is the high-altitude littering event, the second device sends a high-altitude littering prompt information to the first device, and the first device outputs the high-altitude littering prompt information to prompt the user that the high-altitude littering has occurred when receiving the high-altitude littering prompt information sent by the second device. The information fed back by the user is obtained and stored.

[0052] The information fed back by the user is used to indicate whether the recognition result of the second model is incorrect, for example, the information fed back by the user includes correct recognition information and / or incorrect recognition information, the incorrect recognition information is used to indicate that the recognition result of the second model is incorrect, and the correct recognition information is used to indicate that the recognition result of the second model is correct.

[0053] In an embodiment, the manner of running the second model to process the monitoring video data to obtain the second recognition result can be: obtaining a current video frame and a plurality of candidate video frames from the monitoring video data; performing pixel difference on each adjacent two candidate video frames in the plurality of candidate video frames to obtain a plurality of frame difference graphs, and accumulating the plurality of frame difference graphs to obtain an accumulated frame difference graph; determining a foreground target in each candidate video frame, and generating a foreground sequence graph based on the foreground target in each candidate video frame; performing grayscale processing on the current video frame to obtain a grayscale graph, and superimposing the grayscale graph, the accumulated frame difference graph and the foreground sequence graph to obtain a target image; inputting the target image into the second model to obtain the second recognition result.

[0054] In step S103, a target update strategy for the first model and / or the second model is determined according to the second recognition result.

[0055] For example, when the second recognition result is not the high-altitude littering event, the target update strategy is determined to be a preset first update strategy, and the first update strategy is used to update the first model.

[0056] It can be seen that in this example, when the first recognition result is the high-altitude littering event and the second recognition result is not the high-altitude littering event, it can be determined that the recognition result of the first model is incorrect. Therefore, by determining the target update strategy to be the first update strategy used to update the first model, the iterative update of the first model can be realized, and the recognition accuracy of the first model for the high-altitude littering event can be improved.

[0057] Exemplarily, when the second recognition result is the high-altitude littering event, if the recognition error information of the user feedback of the high-altitude littering event is obtained, it is determined that the target update strategy is a preset second update strategy, and the second update strategy is used to update the second model. Alternatively, it is determined that the target update strategy is a preset third update strategy, and the third update strategy is used to update the first model and the second model.

[0058] The first update strategy, the second update strategy, and the third update strategy are different.

[0059] It can be seen that in the present example, when the first recognition result and the second recognition result are the high-altitude littering event, and the user feedback information is the recognition error information, it can be determined that the recognition results of the first model and the second model are incorrect. Therefore, by determining the target update strategy as the third update strategy used to update the first model and the second model, the iterative update of the first model and the second model can be realized, and the recognition accuracy of the first model and the second model for the high-altitude littering event can be improved.

[0060] In step S104, the first model and / or the second model are iteratively updated according to the target update strategy.

[0061] The target update strategy can be the first update strategy, the second update strategy, the third update strategy, and the fourth update strategy. The first update strategy is used to update the first model, the second update strategy is used to update the second model, the third update strategy is used to update the first model and the second model, and the fourth update strategy is used to instruct the high-altitude littering detection system to iteratively update the first model based on the meteorological information at the collection area of the video monitoring data.

[0062] In an embodiment, as shown in Figure 3 Step S104 includes sub-step S1041 to sub-step S1042.

[0063] In step S1041, when the target update strategy is the first update strategy, the first training sample data is generated according to the monitoring video data.

[0064] Exemplarily, the reference high-altitude littering image and the reference background image are obtained from the monitoring video data. The reference high-altitude littering image includes high-altitude littering, and the reference background image does not include high-altitude littering. A plurality of positive example samples are generated according to the reference high-altitude littering image. The positive example samples include high-altitude littering images and first labels. A plurality of negative example samples are generated according to the reference background image. The negative example samples include background images and second labels. The plurality of positive example samples and the plurality of negative example samples are merged to obtain the first training sample data.

[0065] The first label is used to represent that the classification result of the high-altitude projectile image is a high-altitude projectile event, that is, a high-altitude projectile occurs, and the second label is used to represent that the classification result of the background image is a non-high-altitude projectile event, that is, no high-altitude projectile occurs.

[0066] For example, the manner of generating a plurality of positive example samples according to the reference high-altitude projectile image can be: determining the similarity between the reference high-altitude projectile image and each preset high-altitude projectile image in the preset high-altitude projectile image library; obtaining, from the preset high-altitude projectile image library, a preset high-altitude projectile image with a similarity greater than or equal to a preset similarity threshold as a candidate high-altitude projectile image; and combining each candidate high-altitude projectile image with the first label to obtain a plurality of positive example samples. The preset similarity threshold can be set by the user, and the embodiment is not limited in this regard. For example, the preset similarity threshold is 0.9.

[0067] For example, the manner of generating a plurality of negative example samples according to the reference background image can be: determining the similarity between the reference background image and each preset background image in the preset background image library; obtaining, from the preset background image library, a preset background image with a similarity greater than or equal to a preset similarity as a candidate background image; and combining each candidate background image with the second label to obtain a plurality of negative example samples.

[0068] The preset similarity can be set by the user, and the embodiment is not limited in this regard. For example, the preset similarity is 0.85.

[0069] It can be seen that, in the present example, the reference high-altitude projectile image and the reference background image are obtained from the monitoring video data, the image similar to the reference background image is selected from the preset background image library as a candidate background image, the image similar to the reference high-altitude projectile image is selected from the preset high-altitude projectile image library as a candidate high-altitude projectile image, each candidate high-altitude projectile image is combined with the first label to obtain a plurality of positive example samples, each candidate background image is combined with the second label to obtain a plurality of negative example samples, and finally the plurality of positive example samples and the plurality of negative example samples are merged to obtain the first training sample data, which can accurately represent the scene corresponding to the recognition error, so that after the first model is iteratively updated using the first training sample data, the first model can recognize the scene corresponding to the recognition error, thereby improving the recognition accuracy of the first model for the high-altitude projectile event.

[0070] Step S1042, iteratively updating the first model according to the first training sample data;

[0071] Exemplarily, a positive example sample or a negative example sample is obtained from the first training sample data as a target sample; an image in the target sample is input into the first model to obtain a predicted label of the image in the target sample; a model loss value is determined according to a real label in the target sample and the predicted label; when the model loss value is greater than a loss value threshold, the parameters of the first model are updated, and then the step of obtaining the positive example sample or the negative example sample from the first training sample data as the target sample is performed again until the model loss value is less than or equal to the loss value threshold, so as to obtain the first model updated iteratively.

[0072] It can be seen that, in this example, the first model is updated iteratively by using the first training sample data, so as to improve the recognition accuracy of the first model for the high-altitude projectile event.

[0073] Exemplarily, the positive and negative sample ratio of the first training sample data is determined; when the positive and negative sample ratio of the first training sample data does not satisfy a first preset ratio, the first training sample data is adjusted so that the positive and negative sample ratio of the adjusted first training sample data satisfies the first preset ratio.

[0074] It can be seen that, in this example, the positive and negative sample ratio of the first training sample data is adjusted so that the positive and negative sample ratio of the adjusted first training sample data satisfies the first preset ratio, so as to ensure the balance of the positive and negative samples, and ensure that the first model can recognize the scene corresponding to the recognition error after the first model is updated iteratively by using the first training sample data, thereby improving the recognition accuracy of the first model for the high-altitude projectile event.

[0075] The training iteration process of the first model can be completed by the second device, and after the first model is updated iteratively by the second device, the updated first model is sent to the first device, so that the first device deploys the updated first model locally to replace the previous first model.

[0076] In an embodiment, when the target update strategy is the second update strategy, the second training sample data is generated according to the monitoring video data, and the second model is updated iteratively according to the second training sample data. The specific generation manner of the second training sample data can refer to the specific generation manner of the first training sample data, which will not be described herein.

[0077] It can be seen that, in this example, the second training sample data generated by the monitoring video data can accurately represent the scene corresponding to the recognition error of the second model, so as to ensure that the second model can recognize the scene corresponding to the recognition error after the second model is updated iteratively by using the second training sample data, thereby improving the recognition accuracy of the second model for the high-altitude projectile event.

[0078] In an embodiment, when the target updating strategy is the third updating strategy, the first training sample data and the second training sample data are generated according to the monitored video data, the first model is iteratively updated according to the first training sample data, and the second model is iteratively updated according to the second training sample data.

[0079] The first preset ratio and the sample number of the first training sample data can be set by the user, and the embodiment does not make specific limitations thereon. For example, the sample number of the first training sample data is 15000, and the first preset ratio is 1:3.

[0080] For example, the positive and negative sample ratio of the second training sample data is determined, and when the positive and negative sample ratio of the second training sample data does not satisfy the second preset ratio, the second training sample data is adjusted so that the positive and negative sample ratio of the adjusted second training sample data satisfies the second preset ratio.

[0081] The second preset ratio and the sample number of the second training sample data can be set by the user, and the embodiment does not make specific limitations thereon. For example, the sample number of the second training sample data is 20000, and the second preset ratio is 1:1.

[0082] It can be seen that in the present example, the first model is iteratively updated by the first training sample data which can accurately represent the scene corresponding to the recognition error of the first model, and the second model is iteratively updated by the second training sample data which can accurately represent the scene corresponding to the recognition error of the second model, so that the first model and the second model can recognize the scene corresponding to the recognition error, thereby improving the recognition accuracy of the first model and the second model for the high-altitude throwing event.

[0083] In an embodiment, when the second recognition result is not the high-altitude throwing event, the weather information at the collection area of the video monitoring data is obtained, and the preset weather condition required for running the first model is obtained; when the weather information does not satisfy the preset weather condition, it is determined that the target updating strategy is a preset fourth updating strategy; and the first model is iteratively updated according to the fourth updating strategy.

[0084] The fourth updating strategy is used to instruct the high-altitude throwing detection system to iteratively update the first model based on the weather information at the collection area of the video monitoring data, and the preset weather condition can be set based on actual conditions, and the embodiment does not make specific limitations thereon.

[0085] It can be seen that in the present example, when the weather information does not satisfy the preset weather condition required for running the first model, the first model is iteratively updated based on the fourth updating strategy, so that the iteratively updated first model can adapt to the weather condition of the area where the video monitoring data is located, thereby improving the recognition accuracy of the first model for the high-altitude throwing event.

[0086] Exemplarily, the manner of acquiring the weather information at the collection area of the video monitoring data can be: acquiring an identity recognition code of the monitoring device from the video monitoring data; querying a preset mapping relationship between the identity recognition code and the geographic location information to acquire the geographic location information corresponding to the identity recognition code in the video monitoring data; and acquiring the weather information of the area where the monitoring device is located from the weather database based on the geographic location information. The collection area of the video monitoring data is the area where the monitoring device is located, and the preset weather condition can be set based on the actual situation, which is not limited in the embodiment.

[0087] Exemplarily, the manner of iteratively updating the first model according to the fourth update strategy can be: generating first training sample data according to the monitoring video data; adjusting each positive sample in the first training sample data according to the weather information to obtain new positive samples, and adjusting each negative sample in the first training sample data according to the weather information to obtain new negative samples; merging the new positive samples and the new negative samples to obtain fourth training sample data; and iteratively updating the first model according to the fourth training sample data.

[0088] Please refer to Figure 4 , Figure 4 is a schematic diagram of the iterative updating process of the first model in the embodiment of the application. As shown in Figure 8 , the iterative updating process of the first model includes steps S11-S19.

[0089] Step S11, acquiring the collected monitoring video data, and running the first model to process the monitoring video data to obtain a first recognition result;

[0090] Step S12, when it is determined that the first recognition result is a high-altitude throwing event, running the second model to process the monitoring video data to obtain a second recognition result;

[0091] Step S13, when the second recognition result is not a high-altitude throwing event, acquiring a reference high-altitude throwing image and a reference background image from the monitoring video data;

[0092] Step S14, generating a plurality of positive samples according to the reference high-altitude throwing image, and generating a plurality of negative samples according to the reference background image, and merging the plurality of positive samples and the plurality of negative samples to obtain first training sample data;

[0093] Step S15, acquiring a positive sample or a negative sample from the first training sample data as a target sample;

[0094] Step S16, inputting the image in the target sample into the first model to obtain a predicted label of the image in the target sample;

[0095] Step S17: Determine the model loss value based on the true label in the target sample and the predicted label;

[0096] Step S18: When the model loss value is greater than the loss value threshold, update the parameters of the first model and return to execute step S15;

[0097] Step S19: Stop training the first model when the model loss value is less than or equal to the loss value threshold.

[0098] As can be seen in this example, when the first identification result is a high-altitude object throwing event but the second identification result is not, it can be determined that the identification result of the first model is incorrect. Therefore, by obtaining a reference high-altitude object throwing image and a reference background image from the surveillance video data, generating multiple positive samples based on the reference high-altitude object throwing image, and generating multiple negative samples based on the reference background image, and merging the multiple positive samples and multiple negative samples, the first training sample data that accurately represents the scene corresponding to the first model's identification error can be obtained. Then, positive or negative samples are obtained from the first training sample data as target samples, and the images in the target samples are input into the first model to obtain the predicted labels of the images in the target samples. Then, the model loss value is determined based on the true label in the target sample and the predicted label. When the model loss value is greater than the loss value threshold, the parameters of the first model are updated, and the execution of step S15 is returned. When the model loss value is less than or equal to the loss value threshold, the training of the first model is stopped. This allows for iterative updates of the first model, thereby improving the accuracy of the first model in identifying high-altitude object throwing events.

[0099] For example, a monitoring device 100 deployed in a residential community collects video data from the buildings within the community. The monitoring device then sends this video data to a first device 200 in a high-altitude object throwing detection system. The first device 200 can then extract information such as... from the video data. Figure 5 The current video frame and 20 candidate video frames are shown. Then, the pixel difference between any two adjacent candidate video frames is calculated to obtain multiple frame difference maps. These multiple frame difference maps are then accumulated to obtain the following result: Figure 6 The cumulative frame difference map shown is obtained by analyzing... Figure 6 The cumulative frame difference map shown and Figure 5 The current video frame shown is synthesized, that is... Figure 6 The white dashed line 21 in the middle and Figure 5 By combining the current video frames shown, we can obtain the following: Figure 7 The composite image shown will Figure 7 The synthesized image shown is input into the first model for processing, and the result is as follows: Figure 8 The image shown is as follows: Figure 8As shown, the parabolic trajectory 11 is framed by the rectangular frame 12, that is, the first model identifies the parabolic trajectory, and the first identification result is the high-altitude parabolic event. At this time, the first device 200 sends the monitoring video data to the second device 300, the second device 300 runs the second model to process the monitoring video data, and obtains the second identification result. When the second identification result is not the high-altitude parabolic event, it can be determined that the first model deployed in the first device 200 has a high-altitude event identification error. Therefore, the first training sample data can be generated based on the monitoring video data, and the first model can be iteratively updated based on the first training sample data, so as to improve the identification accuracy of the first model for the high-altitude parabolic event.

[0100] Please refer to Figure 9 , Figure 9 is another step flow diagram of the model updating method for high-altitude parabolic event provided by the embodiment of the application.

[0101] As Figure 9 shown, the model updating method for high-altitude parabolic event can include steps S201 to S204.

[0102] Step S201, acquiring the collected monitoring video data, and running the first model to process the monitoring video data to obtain the first identification result.

[0103] For example, the current video frame and a plurality of candidate video frames are acquired from the monitoring video data; pixel difference is performed on each adjacent two candidate video frames in the plurality of candidate video frames to obtain a plurality of frame difference graphs, and the plurality of frame difference graphs are accumulated to obtain an accumulated frame difference graph; the accumulated frame difference graph and the current video frame are synthesized to obtain a synthesized image, and the synthesized image is input into the first model for processing to obtain the first identification result.

[0104] The first identification result includes the high-altitude parabolic event and the non-high-altitude parabolic event, the current video frame is a video frame corresponding to a current system time at a collection time, and the plurality of candidate video frames include the current video frame and a plurality of historical video frames. The historical video frame is a video frame corresponding to a time before the current system time.

[0105] Step S202, when the first identification result is determined to be the high-altitude parabolic event, acquiring the meteorological information at the collection area of the video monitoring data, and acquiring the preset meteorological condition required for running the first model.

[0106] For example, the way to acquire the meteorological information at the collection area of the video monitoring data can be: acquiring the identity recognition code of the monitoring device from the video monitoring data; querying the mapping relationship between the preset identity recognition code and the geographic location information to acquire the geographic location information corresponding to the identity recognition code in the video monitoring data; and acquiring the meteorological information of the area where the monitoring device is located from the meteorological database based on the geographic location information.

[0107] The collection area of the video monitoring data is an area where the monitoring device is located, and the preset meteorological condition can be set based on actual conditions, which is not limited in the embodiment.

[0108] In step S203, when the meteorological information does not satisfy the preset meteorological condition, third training sample data is generated according to the meteorological information.

[0109] For example, the third training sample data can be generated according to the meteorological information in the following manner: obtaining historical training sample data of the first model, wherein the historical training sample data includes a plurality of historical positive samples and a plurality of historical negative samples; adjusting the high-altitude parabolic image in each historical positive sample according to the meteorological information to obtain a new positive sample; adjusting the background image in each historical negative sample according to the meteorological information to obtain a new negative sample; and merging the new positive sample and the new negative sample to obtain the third training sample data.

[0110] The meteorological information includes foggy days, rainy days, snowy days, etc.

[0111] In step S204, the first model is iteratively updated according to the third training sample data.

[0112] For example, the third training sample data includes a plurality of positive samples and a plurality of negative samples. The first model can be iteratively updated according to the third training sample data in the following manner: obtaining a positive sample or a negative sample from the third training sample data as a target sample; inputting the image in the target sample into the first model to obtain a predicted label of the image in the target sample; determining a model loss value according to the real label in the target sample and the predicted label; when the model loss value is greater than a loss value threshold, updating the parameters of the first model, and then returning to the step of obtaining a positive sample or a negative sample from the third training sample data as a target sample until the model loss value is less than or equal to the loss value threshold, thereby obtaining an iteratively updated first model.

[0113] In an embodiment, after the first model is iteratively updated according to the third training sample data, when the meteorological information at the collection area of the video monitoring data satisfies the preset meteorological condition, the first model obtained after the iteration is restored to the first model before the iteration.

[0114] As can be seen, in the example, the updated detection model is restored to the detection model before the update, which can avoid the device running a large detection model all the time and improve the running efficiency.

[0115] In an embodiment, a plurality of first models are stored in the high-altitude litter detection system, and each first model corresponds to different weather conditions. Therefore, after obtaining weather information at the collection area of the video monitoring data, it is determined whether there is a first model in the high-altitude litter detection system that matches the weather information. If there is a first model that matches the weather information, the first model that matches the weather information is run to process the video monitoring data to obtain a first recognition result.

[0116] It can be seen that in this example, by deploying a plurality of first models in the high-altitude litter detection system, and each first model corresponds to different weather information, the first model that matches the weather information can be adaptively selected to process the monitoring video data collected under the corresponding weather condition, which can improve the accuracy of high-altitude litter detection.

[0117] The model updating method for high-altitude litter provided by the embodiment of the application can update the first model based on the weather information when the weather information at the collection area of the video monitoring data does not satisfy the preset weather condition required for running the first model, so that the updated first model can detect high-altitude litter events under new weather conditions, greatly improving the application range and accuracy of the high-altitude litter detection model.

[0118] Please refer to Figure 10 , Figure 10 is a structural schematic block diagram of a high-altitude litter detection system provided by an embodiment of the application.

[0119] As shown in Figure 10 , the high-altitude litter detection system 400 includes a first device 410 and a second device 420, the first device 410 is in communication connection with the second device 420, the first device 410 includes a first model for identifying a high-altitude litter event, and the second device 420 includes a second model for identifying a high-altitude litter event, wherein:

[0120] The first device 410 is configured to obtain monitoring video data, run the first model to process the monitoring video data, and obtain a first recognition result;

[0121] The first device 410 is further configured to, when the first recognition result is determined to be a high-altitude litter event, send the monitoring video data to the second device 420;

[0122] The second device 420 is configured to, after receiving the monitoring video data, run the second model to process the monitoring video data, and obtain a second recognition result;

[0123] The second device 420 is further configured to determine a target update strategy for the first model and / or the second model according to the second recognition result;

[0124] The second device 420 is further configured to perform iterative updating on the first model and / or the second model according to the target updating strategy.

[0125] In an embodiment, the second device 420 is further configured to:

[0126] when the second recognition result is not the high-altitude litter throwing event, determining that the target updating strategy is a preset first updating strategy, and the first updating strategy is used to update the first model;

[0127] when the second recognition result is the high-altitude litter throwing event, if the user feedback recognition error information of the high-altitude litter throwing event is obtained, determining that the target updating strategy is a preset second updating strategy, and the second updating strategy is used to update the second model, or determining that the target updating strategy is a preset third updating strategy, and the third updating strategy is used to update the first model and the second model.

[0128] In an embodiment, the second device 420 is further configured to:

[0129] The iterative updating on the first model and / or the second model according to the target updating strategy comprises:

[0130] when the target updating strategy is the first updating strategy, generating first training sample data according to the monitoring video data, and performing iterative updating on the first model according to the first training sample data;

[0131] when the target updating strategy is the second updating strategy, generating second training sample data according to the monitoring video data, and performing iterative updating on the second model according to the second training sample data;

[0132] when the target updating strategy is the third updating strategy, generating first training sample data and second training sample data according to the monitoring video data, performing iterative updating on the first model according to the first training sample data, and performing iterative updating on the second model according to the second training sample data.

[0133] In an embodiment, the second device 420 is further configured to:

[0134] obtaining a reference high-altitude litter throwing image and a reference background image from the monitoring video data;

[0135] generating a plurality of positive example samples according to the reference high-altitude litter throwing image, wherein the positive example samples comprise high-altitude litter throwing images and first labels;

[0136] generate a plurality of negative example samples according to the reference background image, wherein the negative example samples comprise background images and second labels;

[0137] merge the plurality of positive example samples and the plurality of negative example samples to obtain the first training sample data.

[0138] In an embodiment, the second device 420 is further configured to:

[0139] determine a similarity between the reference high-altitude throwing image and each preset high-altitude throwing image in a preset high-altitude throwing image library;

[0140] obtain, as a candidate high-altitude throwing image, a preset high-altitude throwing image with a similarity greater than or equal to a preset similarity threshold from the preset high-altitude throwing image library;

[0141] combine each candidate high-altitude throwing image with the first label to obtain a plurality of positive example samples.

[0142] In an embodiment, the second device 420 is further configured to:

[0143] determine a positive-negative sample ratio of the first training sample data;

[0144] when the positive-negative sample ratio does not satisfy a first preset ratio, adjust the first training sample data so that a positive-negative sample ratio of the adjusted first training sample data satisfies the first preset ratio.

[0145] In an embodiment, the second device 420 is further configured to:

[0146] obtain meteorological information at a collection area of the video monitoring data, and obtain a preset meteorological condition required for running the first model;

[0147] when the meteorological information does not satisfy the preset meteorological condition, generate third training sample data according to the meteorological information;

[0148] update the first model iteratively according to the third training sample data.

[0149] In an embodiment, the second device 420 is further configured to:

[0150] obtain meteorological information at a collection area of the video monitoring data, and obtain a preset meteorological condition required for running the first model;

[0151] when the meteorological information does not satisfy the preset meteorological condition, generate third training sample data according to the meteorological information;

[0152] According to the third training sample data, the first model is iteratively updated;

[0153] When the meteorological information at the collection area of the video monitoring data meets the preset meteorological condition, the first model obtained after iteration is restored to the first model before iteration.

[0154] The first device 410 can include a notebook computer, a personal computer (PC), an edge detection device, a server, etc., and the second device 420 can include a notebook computer, a personal computer (PC), an edge detection device, a server, etc. The server can be a standalone server, a server cluster composed of multiple servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. For example, the first device 410 is a personal computer, and the second device 420 is a server. For another example, the first device 410 and the second device 420 are both servers.

[0155] It should be noted that the skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the high-altitude throwing detection system described above can refer to the corresponding process in the foregoing model updating method embodiment of the high-altitude throwing, which will not be repeated here.

[0156] The embodiment of the application further provides a storage medium for computer readable storage, the storage medium stores one or more programs, the one or more programs can be executed by one or more processors to implement the steps of the model updating method of the high-altitude throwing provided by any one of the embodiments of the application.

[0157] The storage medium can be an internal storage unit of the high-altitude throwing detection system, such as a hard disk or a memory of the high-altitude throwing detection system. The storage medium can also be an external storage device of the high-altitude throwing detection system, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0158] Those skilled in the art can understand that all or some of the steps in the methods disclosed above, the functional modules / units in the systems and devices can be implemented by software, firmware, hardware, or a combination thereof. In hardware embodiments, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer-readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and can include any information delivery media.

[0159] It should be understood that the term "and / or" as used herein refers to any combination of associated listed items, and all possible combinations, and includes these combinations. It should be noted that the terms "comprising", "including", or any other variant thereof, are intended to cover non-exclusive inclusion, so that processes, methods, articles, or systems including a series of elements not only include those elements, but also include other elements not explicitly listed, or other elements inherent to such processes, methods, articles, or systems. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or system including the element.

[0160] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for updating a model of high-altitude parabolic objects, characterized in that, An application to a high-altitude object throwing detection system, the high-altitude object throwing detection system including a first model and a second model for identifying high-altitude object throwing events, the method comprising: Acquire the collected surveillance video data, and run the first model to process the surveillance video data to obtain the first recognition result; When the first identification result is determined to be a high-altitude object throwing event, the second model is run to process the surveillance video data to obtain a second identification result; Based on the second identification result, a target update strategy is determined for the first model and / or the second model; According to the target update strategy, the first model and / or the second model are iteratively updated; The step of running the first model to process the surveillance video data and obtain a first recognition result includes: The current video frame and multiple candidate video frames are obtained from the monitoring video data; the pixel difference between each two adjacent candidate video frames is calculated to obtain multiple frame difference maps, and the multiple frame difference maps are accumulated to obtain a cumulative frame difference map; the cumulative frame difference map is combined with the current video frame to obtain a composite image, and the composite image is input into the first model for processing to obtain a first recognition result; The method further includes: Obtain meteorological information at the collection area of ​​the monitoring video data, and obtain the preset meteorological conditions that need to be met to run the first model; When the meteorological information does not meet the preset meteorological conditions, a third training sample data is generated based on the meteorological information, and the first model is iteratively updated based on the third training sample data. When the meteorological information at the area where the monitoring video data is collected meets the preset meteorological conditions, the first model obtained after iterative update is restored to the first model before iterative update.

2. The method according to claim 1, characterized in that, The step of determining the target update strategy for the first model and / or the second model based on the second identification result includes: When the second identification result is not a high-altitude object throwing event, the target update strategy is determined to be the preset first update strategy, which is used to update the first model. When the second identification result is a high-altitude object throwing incident, if user feedback on the high-altitude object throwing incident is obtained, the target update strategy is determined to be a preset second update strategy, which is used to update the second model; or, the target update strategy is determined to be a preset third update strategy, which is used to update the first model and the second model.

3. The method according to claim 1, characterized in that, The iterative update of the first model and / or the second model according to the target update strategy includes: When the target update strategy is the first update strategy, first training sample data is generated based on the surveillance video data, and the first model is iteratively updated based on the first training sample data. When the target update strategy is the second update strategy, second training sample data is generated based on the surveillance video data, and the second model is iteratively updated based on the second training sample data; When the target update strategy is the third update strategy, first training sample data and second training sample data are generated based on the monitoring video data, and the first model is iteratively updated based on the first training sample data and the second model is iteratively updated based on the second training sample data.

4. The method according to claim 3, characterized in that, The step of generating the first training sample data based on the surveillance video data includes: Obtain a reference high-altitude projectile image and a reference background image from the surveillance video data; Multiple positive examples are generated based on the reference high-altitude projectile image, wherein the positive examples include the high-altitude projectile image and a first label; Multiple negative examples are generated based on the baseline background image, wherein the negative examples include the background image and a second label; The first training sample data is obtained by merging multiple positive samples and multiple negative samples.

5. The method according to claim 4, characterized in that, The generation of multiple positive samples based on the reference high-altitude parabolic image includes: Determine the similarity between the reference high-altitude projectile image and each preset high-altitude projectile image in the preset high-altitude projectile image library; From the preset high-altitude projectile image library, obtain preset high-altitude projectile images with a similarity greater than or equal to a preset similarity threshold as candidate high-altitude projectile images; The candidate images of objects thrown from high altitudes are combined with the first label to obtain multiple positive sample images.

6. The method according to claim 3, characterized in that, The method further includes: Determine the ratio of positive to negative samples in the first training sample data; When the ratio of positive to negative samples does not meet the first preset ratio, the first training sample data is adjusted so that the ratio of positive to negative samples in the adjusted first training sample data meets the first preset ratio.

7. A high-altitude object throwing detection system, characterized in that, The high-altitude object throwing detection system includes a first device and a second device, the first device being communicatively connected to the second device. The first device includes a first model for identifying high-altitude object throwing events, and the second device includes a second model for identifying high-altitude object throwing events, wherein: The first device is used to acquire surveillance video data, run the first model to process the surveillance video data, and obtain a first recognition result; The first device is also used to send the surveillance video data to the second device when it is determined that the first identification result is a high-altitude object throwing event; The second device is used to process the surveillance video data by running the second model after receiving the surveillance video data, and to obtain a second recognition result; The second device is further configured to determine a target update strategy for the first model and / or the second model based on the second identification result; The second device is further configured to iteratively update the first model and / or the second model according to the target update strategy; The step of running the first model to process the surveillance video data and obtain a first recognition result includes: The current video frame and multiple candidate video frames are obtained from the monitoring video data; the pixel difference between each two adjacent candidate video frames is calculated to obtain multiple frame difference maps, and the multiple frame difference maps are accumulated to obtain a cumulative frame difference map; the cumulative frame difference map is combined with the current video frame to obtain a composite image, and the composite image is input into the first model for processing to obtain a first recognition result; The second device is also used for: Obtain meteorological information at the collection area of ​​the monitoring video data, and obtain the preset meteorological conditions that need to be met to run the first model; When the meteorological information does not meet the preset meteorological conditions, a third training sample data is generated based on the meteorological information, and the first model is iteratively updated based on the third training sample data. When the meteorological information at the area where the monitoring video data is collected meets the preset meteorological conditions, the first model obtained after iterative update is restored to the first model before iterative update.

8. A storage medium, characterized in that, For computer-readable storage, the storage medium stores one or more programs that can be executed by one or more processors to implement the model update method for high-altitude projectiles as described in any one of claims 1-6.

Images