Integrated analysis method and system suitable for multi-scene dynamic image, storage medium and program product

Abstract

The invention provides an integrated analysis method and system suitable for a multi-scene dynamic image, a storage medium and a program product, and relates to the technical field of image equipment. And tracking the heterozoon body. In the tracking process, original volume information of the different animal bodies in the actual space is calculated, and when the condition that the different animal bodies cannot be tracked occurs, all other different animal bodies in the monitoring picture can be obtained, and volume calculation strategies are carried out on the different animal bodies respectively. And then, by judging whether the difference value between the sum of the volume information and the original volume information is smaller than a difference value threshold value or not, possible protozoon and heterozoon bodies are screened out. And if the difference value meets the requirement, determining other heterozoon bodies of which the volume sequence change trend is reduced as protoheterozoon bodies, and tracking the protoheterozoon bodies. The method effectively makes up for the defects that a related tracking mode is easy to lose a target and is easy to misjudge, the target can be retrieved by virtue of volume-related judgment after the target is lost, the whole tracking process can be smoothly continued, and the tracking success rate of illegal vehicles with muck poured disorderly is increased.

Description

Technical Field

[0001] The present application relates to the field of image device technology, and in particular to an integrated analysis method, system, storage medium and program product applicable to multi-scene dynamic images. Background Art

[0002] Nowadays, the illegal behavior of dumping construction waste has been repeatedly banned but continues to occur, which has brought serious negative impacts on many aspects of the city's ecological environment, urban appearance, infrastructure and so on.

[0003] There are many problems in the scenario of tracking illegal vehicles that dump waste. First of all, the waste often partially obscures the main structure of the vehicle, which makes it difficult to identify the various features of the vehicle that were originally clearly identifiable. The most important thing is that when dumping waste, the vehicle will undergo extremely significant deformation, such as distortion of the body structure and change of the angle of the car body. As a result, the characteristics of the vehicle change extremely drastically. Even in a very short period of time, the characteristics presented by the front and back of the vehicle are difficult to match, which undoubtedly adds great difficulty to the monitoring work. If you try to shorten the sampling time interval in order to capture the changes in vehicle characteristics more accurately, new problems will arise. Because this means that the entire monitoring system needs to have higher performance, both in terms of hardware materials and computing resources, which is too costly for the specific scenario of tracking the dumping of waste.

[0004] In summary, the relevant technology has a poor success rate in tracking illegal vehicles that dump waste. Summary of the invention

[0005] The present application provides an integrated analysis method, system, storage medium and program product applicable to multi-scene dynamic images, which are used to improve the success rate of tracking illegal vehicles that dump waste indiscriminately while the cost increase is controllable.

[0006] In a first aspect, the present application provides an integrated analysis method applicable to multi-scene dynamic images, comprising: acquiring monitoring images of different scenes by means of image acquisition devices arranged in the multiple scenes; analyzing a preset area of ​​the acquired monitoring image to determine whether there is an alien object in the preset area, wherein the alien object refers to an object whose elements in the preset area differ from other areas of the monitoring image by more than a preset threshold; dividing the alien object into a plurality of first sub-areas, and clustering the first sub-areas according to similarity and interval to obtain a plurality of second sub-areas; wherein the higher the similarity or the lower the interval, the higher the probability of clustering; and according to a height ratio threshold, The foreign object is divided into an upper area and a lower area, and the second sub-area in the upper area is placed into the algorithm warehouse in order of shape complexity from large to small to determine whether it has pre-stored slag features; the second sub-area in the lower area is placed into the algorithm warehouse in order of shape complexity from small to large to determine whether it has pre-stored automobile parts features; when any second sub-area in the upper area has pre-stored slag features, and any second sub-area in the lower area has pre-stored automobile parts features, the foreign object is tracked; when tracking, other image acquisition devices of the corresponding scene are awakened to acquire packets A plurality of target images containing alien objects, other image acquisition devices are arranged at different positions of the corresponding scenes, have different shooting angles, and start a volume calculation strategy; the volume calculation strategy includes: calculating the original volume information of the alien object in the actual space according to the height, angle, focal length of the other image acquisition devices and the pixel size and position information occupied by the alien object in the target image; periodically updating the volume data of the alien object at time intervals to form a volume sequence of the volume change of the alien object over time; in the case where the alien object cannot be tracked, obtaining all other alien objects in the monitoring image; performing the volume calculation strategy on the other alien objects respectively to obtain other volume information and other volume sequences; selecting two other volume information from all other volume information and determining them as the first other volume information and the second other volume information; judging whether the difference between the sum of the first other volume information and the second other volume information and the original volume information is less than the difference threshold; if it is less than the difference threshold, determining the change trend of the other volume sequence corresponding to the first other volume information and the second other volume information; determining the other alien objects whose change trend of the other volume sequence corresponding to the first other volume information or the second other volume information is decreasing as alien objects, and tracking them, and changing the original volume information with the corresponding volume information.

[0007] By adopting the above technical solution, the monitoring screens of different scenes are obtained by using the image acquisition equipment arranged in multiple scenes, and the foreign objects therein are judged, so as to lock the objects of interest for subsequent processing. Then, the foreign objects are divided into several first sub-areas, and these first sub-areas are clustered according to similarity and interval to obtain several second sub-areas. The vehicles transporting slag are divided into several areas with obvious characteristics, singleness and large area, which is convenient for distinguishing the characteristic parts of the vehicles and slag, and improving the accuracy and efficiency of the overall judgment. Considering that the slag is generally above and has the most irregular shape, the second sub-area inside the upper area is placed in the algorithm warehouse in order from large to small according to the complexity of the shape, to determine whether it has the pre-stored slag characteristics; and some parts of the car parts are located below, and the shapes are often relatively regular and simple, so the second sub-area inside the lower area is placed in the algorithm warehouse in order from small to large according to the complexity of the shape, to determine whether it has the pre-stored car parts characteristics. In this way, the feature judgment can be fully based on the distribution characteristics and shape characteristics of the slag and car parts themselves, reducing the difficulty of calculation and resource consumption. Then, the foreign objects are tracked. During tracking, other image acquisition devices of the corresponding scene are awakened to collect several target images containing alien objects. These devices, which are distributed in different positions and have different shooting angles, can collect multi-view images that can present the state of alien objects in all directions, providing a data basis for subsequent volume calculation. Then, the volume calculation strategy is started to calculate the original volume information of the alien object in the actual space, and then the volume data of the alien object is periodically updated at time intervals to form a volume sequence of the volume of the alien object changing over time, so as to grasp the volume change of the alien object in real time and provide a key basis for judging its behavior state. When the alien object cannot be tracked, all other alien objects in the monitoring screen will be obtained, and the volume calculation strategy will be performed on them respectively to obtain other volume information and other volume sequences. After that, two suitable other volume information are selected from all other volume information, which are determined as the first other volume information and the second other volume information. By judging whether the difference between their sum and the original volume information is less than the difference threshold, the possible original alien objects are screened out. If the difference meets the requirements, the change trend of other corresponding volume sequences is further determined, and other alien objects with a decreasing volume sequence change trend are determined as original alien objects and tracked, and the original volume information is modified with the corresponding volume information.In the actual scenario of dumping construction waste, faced with many problems such as the obstruction of the vehicle by the construction waste, the constant change of the vehicle's own shape with the dumping action, and the complex and difficult to identify vehicle features, the system effectively makes up for the defects of related tracking methods that are easy to lose the target and make misjudgments. It can not only accurately lock the target for continuous tracking, but also reduce the consumption of computing resources through reasonable area division and feature judgment. It can also find the target after it is lost by volume-related judgment, so that the entire tracking process can be continued smoothly, increasing the success rate of tracking illegal vehicles that dump construction waste.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the difference threshold is positively correlated with the time elapsed from the loss of tracking of the alien object to the present moment; the step of selecting two other volume information from all other volume information and determining them as the first other volume information and the second other volume information specifically includes: selecting two other volume information whose distance is less than a preset separation threshold from all other volume information, and determining them as the first other volume information and the second other volume information; the preset separation threshold is positively correlated with the time elapsed from the loss of tracking of the alien object to the present moment.

[0009] By adopting the above technical solution, taking into account the influence of dust interference and the passage of time, for example, after the dust floats away, the original volume of the object may change greatly due to changes in the visual situation. As time goes by, the uncertainty increases. At this time, the difference threshold is associated with the duration, which can better accommodate the uncertainty of volume changes caused by dust and time factors, avoid misjudgment due to overly strict volume thresholds, thereby increasing the probability of screening out the original alien object from many other alien objects, and creating favorable conditions for subsequent re-tracking. At the same time, two other volume information with a distance less than the preset separation threshold are selected from all other volume information, and are determined as the first other volume information and the second other volume information, and the preset separation threshold is positively correlated with the time from the loss of tracking of the alien object to the present, taking into account the spatial distance factor that changes with time in the dust interference scene. In the case where dust causes the picture to be blurred and the object features are difficult to clearly distinguish, the vehicle may drive away over time, and the distance between the slag and the vehicle increases. By positively associating the preset separation threshold with the duration, the judgment standard for the distance can be dynamically adjusted according to time. As time goes by, the distance restriction range is reasonably relaxed so that the alien objects corresponding to the two other selected volume information are more consistent with the spatial changes of the original target in a dusty environment and after the vehicle may move, avoiding errors in distance judgment caused by visual illusions caused by dust and distance changes caused by the vehicle leaving, thereby more accurately screening out possible original alien objects and enhancing the accuracy of re-determining the target.

[0010] In combination with some embodiments of the first aspect, in some embodiments, after the step of tracking the alien object, the method also includes: generating a corresponding alarm strategy; judging whether the movement rhythm data of the alien object meets the preset uniform rule; if it meets the preset uniform rule, judging whether the movement rhythm data is within the movement rhythm feature interval in the feature library of litter dumping actions; if it is within the movement rhythm feature interval in the feature library of litter dumping actions, determining it as litter dumping and activating the alarm strategy; if it is not within the movement rhythm feature interval in the feature library of litter dumping actions, returning to the step of tracking the alien object until the preset time is reached or there is no alien object on the monitoring screen.

[0011] By adopting the above technical solution, when it is determined to be a slag transport vehicle, an alarm strategy is pre-generated to improve the efficiency of subsequent alarms and further obtain its movement rhythm data. By judging whether the movement rhythm data meets the preset uniformity rules, those cases where similar features appear accidentally but are not actually dumping slag are screened out. Even if it meets the uniformity rules, it is still necessary to continue to judge whether it is within the movement rhythm feature interval in the feature library of slag dumping actions. Only when it is confirmed layer by layer that it meets the requirements, the alarm strategy is activated. This makes the judgment of the behavior of dumping slag more accurate, effectively avoids the misjudgment caused by the similarity between some normal transportation behaviors and the characteristics of dumping slag, and reduces the manpower and time resources consumed by supervisors to verify the misjudgment.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after the step of determining whether the movement rhythm data conforms to a preset uniform rule, the method further includes: if it does not conform to the preset uniform rule, collecting all behavior data of the alien object from the time it appears on the monitoring screen to the current moment, the behavior data including: driving trajectory, stop duration, speed change, turning frequency; clustering the behavior data with various preset normal behavior pattern data; if the behavior data is clustered by any normal behavior pattern data, deleting the alarm strategy; if the behavior data is not clustered by any normal behavior pattern data, returning to the step of tracking the alien object and obtaining the movement rhythm data of the alien object until the preset time is reached or there is no alien object on the monitoring screen.

[0013] By adopting the above technical solution, when it is determined that the movement rhythm data does not meet the preset uniform rules, the multi-dimensional behavior data of the alien object from its appearance to the present is collected, and these data can fully reflect its behavioral characteristics. Then the behavior data is clustered with the preset normal behavior pattern data. Through this comparative clustering, it can be accurately judged whether its behavior is normal. If it is clustered by the normal pattern, it means that it may be a normal situation. Deleting the alarm strategy can avoid false alarms, reduce unnecessary interference, and improve the accuracy of the alarm; if it is not clustered, continue to track and obtain new data, and repeatedly judge until the conditions are met.

[0014] In combination with some embodiments of the first aspect, in some embodiments, when the behavior data is not clustered by any normal behavior pattern data, the step of returning to tracking the alien object to obtain the movement rhythm data of the alien object until a preset time is reached or there is no alien object on the monitoring screen, the method also includes: after the preset time is reached or there is no alien object on the monitoring screen, if the behavior data is not clustered by any normal behavior pattern data; outputting the behavior data to a database to be determined; when the database to be determined is determined, adding a new normal behavior pattern, and determining the behavior data as sub-data of the new normal behavior pattern.

[0015] By adopting the above technical solution, when the behavior data is not clustered and reaches the preset time or there is no foreign matter in the monitoring screen, it is output to the database to be determined. The data is retained for subsequent analysis and further confirmation. When the database to be determined is determined, a new normal behavior pattern is added and the behavior data is used as sub-data, which can continuously expand the scope of normal behavior patterns, making the system more comprehensive in its understanding of normal behavior, and thus more accurately distinguishing normal from abnormal behavior in subsequent judgments.

[0016] In combination with some embodiments of the first aspect, in some embodiments, if the movement rhythm feature interval is not within the movement rhythm feature interval in the feature library of illegal dumping actions, then the step of tracking the alien object is returned until the preset time is reached or there is no alien object in the monitoring screen; when the preset time is reached or there is no alien object in the monitoring screen, and the movement rhythm data is not within the movement rhythm feature interval in the feature library of illegal dumping actions; the alarm strategy is deleted.

[0017] By adopting the above technical solution, if the movement rhythm data is not within the interval of the feature library of landfill dumping, the alarm strategy will be deleted after continuous tracking until the preset time or the condition of no foreign matter is met. This is to take into account that there may be special circumstances that cause the rhythm data to be temporarily abnormal but not landfill dumping behavior. Deleting the alarm strategy can avoid false alarms caused by such circumstances, reduce invalid alarm information, and allow the monitoring system to focus on real landfill dumping behavior.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the second sub-area inside the upper area is placed into the algorithm warehouse in order from large to small in terms of shape complexity, and the step of judging whether there are pre-stored slag features specifically includes: according to the image acquisition device corresponding to the second sub-area, selecting the corresponding algorithm from the algorithm warehouse; according to the algorithm warehouse, allocating computing resources to judge whether there are pre-stored slag features.

[0019] By adopting the above technical solution, after placing the second sub-areas in descending order of shape complexity, the algorithm is selected according to the corresponding image acquisition device, which can match the most suitable algorithm based on the device characteristics and improve the judgment efficiency. Then, computing resources are allocated according to the algorithm warehouse, so that resources can be allocated on demand to avoid resource waste.

[0020] In a second aspect, the present application provides an integrated analysis system applicable to multi-scene dynamic images, and the integrated analysis system applicable to multi-scene dynamic images includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and one or more processors call the computer instructions to enable the integrated analysis system applicable to multi-scene dynamic images to perform the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, the present application provides a computer program product comprising instructions, which, when run on an integrated analysis system for multi-scene dynamic images, enables the integrated analysis system for multi-scene dynamic images to perform the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an integrated analysis system for multi-scene dynamic images, enables the integrated analysis system for multi-scene dynamic images to execute the method described in the first aspect and any possible implementation of the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Use the image acquisition equipment arranged in multiple scenes to obtain the monitoring screens of different scenes, and identify the foreign objects therein, so as to lock the objects of interest for subsequent processing. Then, the foreign objects are divided into several first sub-regions, and these first sub-regions are clustered according to similarity and interval to obtain several second sub-regions. The vehicles transporting slag are divided into several regions with obvious characteristics, singleness and large areas, which is convenient for distinguishing the characteristic parts of the vehicles and slag, and improving the accuracy and efficiency of the overall judgment. Considering that slag is generally above and has the most irregular shape, the second sub-region inside the upper region is placed in the algorithm warehouse in order from large to small according to the complexity of the shape to determine whether it has the pre-stored slag characteristics; while some parts of the automobile are located below, and the shapes are often relatively regular and simple, so the second sub-region inside the lower region is placed in the algorithm warehouse in order from small to large according to the complexity of the shape to determine whether it has the pre-stored automobile parts characteristics. In this way, the feature judgment can be fully based on the distribution characteristics and shape characteristics of the slag and automobile parts themselves, reducing the difficulty of calculation and resource consumption. Then, the foreign objects are tracked. During tracking, other image acquisition devices of the corresponding scene are awakened to collect several target images containing alien objects. These devices, which are distributed in different positions and have different shooting angles, can collect multi-view images that can present the state of alien objects in all directions, providing a data basis for subsequent volume calculation. Then, the volume calculation strategy is started to calculate the original volume information of the alien object in the actual space, and then the volume data of the alien object is periodically updated at time intervals to form a volume sequence of the volume of the alien object changing over time, so as to grasp the volume change of the alien object in real time and provide a key basis for judging its behavior state. When the alien object cannot be tracked, all other alien objects in the monitoring screen will be obtained, and the volume calculation strategy will be performed on them respectively to obtain other volume information and other volume sequences. After that, two suitable other volume information are selected from all other volume information, which are determined as the first other volume information and the second other volume information. By judging whether the difference between their sum and the original volume information is less than the difference threshold, the possible original alien objects are screened out. If the difference meets the requirements, the change trend of other corresponding volume sequences is further determined, and other foreign objects with a decreasing trend in volume sequence are determined as original foreign objects, and tracked, and the original volume information is changed with the corresponding volume information. In the actual scene of dumping slag, facing many problems such as the occlusion of the vehicle by the slag, the constant change of the vehicle's own shape with the dumping action, and the complex and difficult to distinguish vehicle features, it effectively makes up for the defects of the relevant tracking methods that are easy to lose the target and easy to misjudge. It can not only accurately lock the target for continuous tracking, but also reduce the consumption of computing resources through reasonable area division and feature judgment. After the target is lost, it can find the target again by volume-related judgment, so that the entire tracking process can be smoothly continued, and the success rate of tracking illegal vehicles that dump slag is increased.

[0024] 2. Considering the influence of dust interference and the passage of time, for example, after the dust floats away, the original volume of the object may change greatly due to changes in the visual situation. As time goes by, the uncertainty increases. At this time, associating the difference threshold with the duration can better accommodate the uncertainty of volume changes caused by dust and time factors, avoid misjudgment due to overly strict volume thresholds, thereby increasing the probability of screening out the original alien object from many other alien objects, and creating favorable conditions for subsequent re-tracking. At the same time, two other volume information with a distance less than the preset separation threshold are selected from all other volume information, and are determined as the first other volume information and the second other volume information. The preset separation threshold is positively correlated with the time from the loss of tracking of the alien object to the present, taking into account the spatial distance factor that changes with time in the dust interference scene. In the case where dust causes the picture to be blurred and the object features are difficult to clearly distinguish, the vehicle may drive away over time, and the distance between the slag and the vehicle increases. By positively associating the preset separation threshold with the duration, the judgment standard for the distance can be dynamically adjusted according to time. As time goes by, the distance restriction range is reasonably relaxed so that the alien objects corresponding to the two other selected volume information are more consistent with the spatial changes of the original target in a dusty environment and after the vehicle may move, avoiding errors in distance judgment caused by visual illusions caused by dust and distance changes caused by the vehicle leaving, thereby more accurately screening out possible original alien objects and enhancing the accuracy of re-determining the target.

[0025] 3. When it is determined to be a slag transport vehicle, an alarm strategy is generated in advance to improve the efficiency of subsequent alarms and further obtain its movement rhythm data. By judging whether the movement rhythm data meets the preset uniformity rules, those cases where similar features appear accidentally but are not actually indiscriminate dumping of slag are screened out. Even if it meets the uniformity rules, it is still necessary to continue to judge whether it is within the movement rhythm feature interval in the feature library of indiscriminate dumping actions. Only when it is confirmed that all the requirements are met, the alarm strategy is activated. This makes the judgment of indiscriminate dumping of slag more accurate, effectively avoids the misjudgment caused by the similarity between some normal transportation behaviors and indiscriminate dumping of slag characteristics, and reduces the manpower and time resources consumed by supervisors to verify misjudgments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of an integrated analysis method applicable to multi-scene dynamic images in an embodiment of the present application; Figure 2 is another flow chart of the integrated analysis method applicable to multi-scene dynamic images in an embodiment of the present application; Figure 3 is another flow chart of the integrated analysis method applicable to multi-scene dynamic images in an embodiment of the present application; Figure 4 It is an exemplary hardware structure diagram of an integrated analysis system suitable for multi-scene dynamic images in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] See also Figure 1 , Figure 1 It is a flow chart of an integrated analysis method applicable to multi-scene dynamic images in an embodiment of the present application; S101, acquiring monitoring images of different scenes through image acquisition devices arranged in multiple scenes; Among them, multiple scenes refer to multiple scenes of different types and environments, covering various areas that may need to be monitored; It should be noted that the present application has the ability to carry out the following steps in multiple scenarios. However, for ease of understanding, the subsequent explanations of steps S102 to S112 are all explained by taking the situation occurring in one scenario as an example.

[0030] In some preferred embodiments, for a scene, only one of the image acquisition devices needs to be activated, and the activated image acquisition device should be the one with the largest shooting area. Because the staged tasks undertaken by step S101 and the subsequent steps S102 to the current step focus on determining whether a foreign object appears in the corresponding scene. By selecting the image acquisition device with the largest shooting area, the scene range can be covered to the greatest extent, and the image information in the scene can be captured more comprehensively and efficiently, thereby providing strong data support for accurately determining whether there is a foreign object, avoiding missing potential target objects due to limited shooting areas, and ensuring that this staged task is successfully completed.

[0031] S102, analyzing a preset area of ​​the acquired monitoring image to determine whether there is a foreign object in the preset area, wherein a foreign object refers to an object whose elements in the preset area differ from other areas of the monitoring image by more than a preset threshold; The preset area refers to a specific area pre-set in the monitoring screen, which is the focus of the analysis, such as a specific rectangular area in the screen. An alien object refers to an object whose elements in the preset area differ from other areas of the monitoring screen by more than a preset threshold. The difference threshold is a pre-set measurement standard used to determine whether an object is abnormal. For example, an object with a certain degree of difference in color, brightness, shape, etc. can be regarded as an alien object.

[0032] In some embodiments, the elements in the area are then feature extracted and analyzed. By comparing the features with those in other areas of the monitoring screen, the degree of difference is calculated. When the difference exceeds a preset difference threshold, it is determined that there is an alien object in the area, providing a basis for further processing of the alien object.

[0033] In some specific embodiments, the monitoring image is first pre-processed using an image processing algorithm, such as noise reduction, contrast enhancement, etc.; then the features of the preset area and other areas are extracted, such as color histogram, texture features, etc.; then the difference between the two is calculated and compared with a preset difference threshold. If the threshold is exceeded, it is determined that an alien object exists.

[0034] It should be noted that when judging whether there are any foreign objects in the preset area, both horizontal comparison and vertical comparison can be used. The so-called horizontal comparison is to compare the preset area with other areas in the same monitoring screen to see if there are any differences. For example, when animals, people and other elements appear in the screen, the preset area where these elements are located will often show obvious differences in color, shape, movement state, etc. compared with other regular areas in the screen, so as to judge whether there are any foreign objects. The vertical comparison is carried out on adjacent monitoring screens, and observes whether there are any feature differences between the preset area and the same area in the adjacent monitoring screens at different times. For example, if a new pile of soil suddenly appears in the subsequent adjacent monitoring screens of an originally flat ground area, it means that the area has changed. Through this comparison with the same area at different times, it can also be judged whether there are any foreign objects in the preset area.

[0035] S103, dividing the alien object into a plurality of first sub-regions, and clustering the first sub-regions according to similarity and interval to obtain a plurality of second sub-regions; wherein the higher the similarity or the lower the interval, the higher the probability of clustering; Among them, the first sub-region is a smaller regional unit obtained by segmenting the alien object. The purpose of segmentation is to analyze the characteristics of the alien object in more detail. The second sub-region is the region obtained by clustering the first sub-region according to similarity and interval. The similarity is used to measure the similarity between two first sub-regions, and the interval represents the spatial distance between two first sub-regions. The higher the similarity or the lower the interval, the higher the probability of clustering.

[0036] In some embodiments, in order to better understand the structure and characteristics of the alien object, it is divided into multiple first sub-regions. Then, by calculating the similarity and interval between each first sub-region, the first sub-regions with high similarity and small interval are clustered into second sub-regions. In this way, the characteristics of the alien object can be classified and integrated, which is convenient for more accurate judgment and identification.

[0037] In some specific embodiments, the alien object is divided into multiple first sub-regions based on a grid; then the feature vector of each first sub-region, such as color feature, shape feature, etc., is calculated; then the similarity and interval are calculated based on the feature vector, and a clustering algorithm is used to cluster the first sub-regions with high similarity and low interval into second sub-regions, which is not limited here.

[0038] In this way, the vehicles transporting the slag can be divided into several areas with obvious characteristics, singleness and large area.

[0039] It should be noted that in some embodiments, the number of first sub-areas will be considered and screened. Because different alien objects have different numbers of first sub-areas, some alien objects have relatively few first sub-areas due to their own characteristics, which often means that the overall volume corresponding to the alien object is relatively small. From a practical point of view, vehicles that do not meet the classification of transporting slag will be judged to have relatively limited value, and will not be further processed. S104, dividing the foreign body into an upper region and a lower region according to the height ratio threshold, placing the second sub-region inside the upper region into the algorithm warehouse in order of shape complexity from large to small, and judging whether it has pre-stored slag features; placing the second sub-region inside the lower region into the algorithm warehouse in order of shape complexity from small to large, and judging whether it has pre-stored automobile parts features; In some embodiments, shape complexity is the ratio of the perimeter of the object's boundary to its area. For example, a pile of dirt with an irregular shape and many bumps will have a very tortuous boundary, and its perimeter value will be much larger than that of a simple shape with the same area. Under the same area, objects with complex shapes have longer perimeters, so by calculating the ratio of perimeter to area, the larger the ratio, the more tortuous its boundary, the more irregular its shape, and the more complex it is.

[0040] Then, for the second sub-region of the upper area, sort them from large to small according to shape complexity. Use the feature extraction algorithm to combine these features into a feature vector. The feature vector is input into the support vector machine classifier in the algorithm warehouse, which has been trained using a large amount of slag sample data. The classifier calculates the similarity between the feature vector and the pre-stored slag feature. If the similarity is greater than the set threshold, it is determined that the second sub-region has slag features. For the second sub-region of the lower area, sort them from small to large according to shape complexity, and also extract feature vectors and input them into another decision tree-based classifier in the algorithm warehouse. The classifier is trained using automobile parts sample data and uses judgment rules to determine whether it has automobile parts features.

[0041] It can be seen that considering that the slag is generally at the top and has the most irregular shape, the second sub-region inside the upper region is placed in the algorithm warehouse in order from large to small in terms of shape complexity to determine whether it has the pre-stored slag features; while some of the car parts are at the bottom and are often relatively regular and simple in shape, the second sub-region inside the lower region is placed in the algorithm warehouse in order from small to large in terms of shape complexity to determine whether it has the pre-stored car parts features. This can fully judge the features based on the distribution characteristics and shape characteristics of the slag and car parts themselves, reducing the difficulty of calculation and resource consumption.

[0042] In some embodiments, step S104 specifically includes: S1041. Filtering a corresponding algorithm from an algorithm warehouse according to an image acquisition device corresponding to the second sub-region; The algorithm warehouse is a database that stores a variety of algorithms for image analysis and feature recognition, including algorithms designed for different scenes, different features, and different image acquisition device characteristics. Screening out the corresponding algorithm means selecting the most suitable algorithm for analyzing and processing the image to identify specific features (such as slag features) from the algorithm warehouse according to the characteristics of the image acquisition device corresponding to the second sub-area.

[0043] In some embodiments, a plurality of different types of image acquisition devices may be deployed in the monitoring system, such as ordinary visible light cameras, thermal imaging cameras, etc., which photograph foreign objects from different angles and in different ways. After obtaining the second sub-area, it is first necessary to determine which image acquisition device the image of the sub-area is obtained by. Then, according to the parameters and characteristics of the image acquisition device, a matching algorithm is searched from the algorithm warehouse.

[0044] S1042. Allocate computing resources according to the algorithm warehouse to determine whether there are pre-stored slag features.

[0045] In some embodiments, after a suitable algorithm is screened out, it is necessary to allocate corresponding CPU time, GPU core number, memory space and other resources from the system's computing resource pool according to the complexity and computing requirements of the algorithm. After the computing resources are allocated, the image of the second sub-area is input into the algorithm for processing. The algorithm extracts features from the image and compares them with pre-stored slag features. By calculating the similarity between the features or using a classification algorithm to make a judgment, if the similarity exceeds a certain threshold or is classified as a slag category, it is determined that the second sub-area has the pre-stored slag features; otherwise, it is determined that it does not have.

[0046] It can be seen that after placing the second sub-area from large to small according to the shape complexity, the algorithm is screened according to the corresponding image acquisition device, which can match the most suitable algorithm based on the device characteristics and improve the judgment efficiency. Then, computing resources are allocated according to the algorithm warehouse, so that resources can be allocated on demand to avoid resource waste.

[0047] S105, when any second sub-region within the upper region has a pre-stored feature of slag, and any second sub-region within the lower region has a pre-stored feature of automobile parts, tracking the foreign object; This step is performed after determining that the second sub-areas of the upper area and the lower area have pre-stored slag features and automobile parts features, respectively. The scenario is to determine that the foreign object may be a vehicle transporting slag, which needs to be continuously monitored.

[0048] In some specific embodiments, a target detection algorithm is used to determine the position of an alien object in the monitoring screen; then a tracking algorithm such as a Kalman filter is used to predict the position of the alien object at the next moment based on its historical position and motion state; finally, the position information of the alien object is continuously updated in subsequent monitoring screens to achieve continuous tracking. It is understandable that this step can also be implemented in other ways, such as using a tracking algorithm based on feature matching, etc., which is not limited here.

[0049] S106, during tracking, waking up other image acquisition devices of the corresponding scene to acquire several target images containing the alien object, the other image acquisition devices are arranged at different positions of the corresponding scene, have different shooting angles, and start a volume calculation strategy; the volume calculation strategy includes: calculating the original volume information of the alien object in the actual space according to the height, angle, focal length of the other image acquisition devices and the pixel size and position information occupied by the alien object in the target image; periodically updating the volume data of the alien object at time intervals to form a volume sequence of the volume of the alien object changing over time; Among them, other image acquisition devices refer to image acquisition devices arranged at different positions of the corresponding scene and with different shooting angles, in addition to the devices that initially acquire the monitoring screen, which are used to obtain information about alien objects from multiple perspectives. The target screen refers to the image or video clip containing the alien object acquired by other image acquisition devices. The volume calculation strategy is a set of methods and processes for calculating the volume of the alien object in the actual space. Height, angle, and focal length are parameters of other image acquisition devices. The pixel size and position information refers to the pixel size and position of the alien object in the target screen. The original volume information is the initial volume value of the alien object in the actual space obtained by calculation. The volume sequence refers to the sequence formed by the volume data of the alien object periodically updated at time intervals, which is used to reflect the change of volume over time.

[0050] In some specific embodiments, first, a three-dimensional coordinate system is established according to the installation position and parameters of other image acquisition devices; then, using the principle of stereoscopic vision, the position and size of the alien object in the three-dimensional coordinate system are calculated through the pixel information of the alien object in multiple target images; then, based on the calculated size information, the original volume information is calculated using a volume calculation formula (such as using the corresponding geometric formula for regular objects, using the voxelization method for irregular objects, etc.); finally, the above steps are repeated at a set time interval to update the volume data and form a volume sequence. Optionally, the structured light three-dimensional scanning technology is first used to emit and receive light through other image acquisition devices to obtain the three-dimensional point cloud data of the alien object; then the three-dimensional point cloud data is processed and analyzed to calculate the volume of the alien object; then the time series analysis method is used to periodically update the volume data at time intervals to form a volume sequence, which is not limited here.

[0051] S107, when it is impossible to track the alien object, obtaining all other alien objects in the monitoring screen; It should be noted that the core principle of the tracking algorithm is to carry out target recognition work first. In practical applications, if a situation occurs in which a foreign object cannot be tracked, it can be determined that target recognition cannot be achieved. The occurrence of this phenomenon is closely related to the special circumstances of vehicles transporting slag when dumping slag. When the vehicle is dumping, extremely significant deformation will occur, which is manifested in the distortion of the body structure and the drastic change of the car body angle. These changes will cause drastic changes in the originally stable and easy-to-identify features of the vehicle. The features presented by the vehicle before and after are very different, making it difficult for the tracking algorithm to continuously and accurately identify the target when the features change so drastically. Therefore, in step S105, the tracking operation is difficult to continue.

[0052] It should be understood that dumping debris is an illegal act, and drivers will remove the license plate before dumping. That is to say, in actual scenarios, it is more about the characteristics of the car itself that are tracked.

[0053] The principle and process of identifying an alien object in this step are similar to those of step S102. The relevant principles and processes can refer to step S102 and are not limited here.

[0054] It should be noted that in actual application scenarios, when the above-mentioned situation of being unable to continue tracking occurs, at least two new objects will appear in the monitoring screen at this time. One is the vehicle. Although it is essentially the same vehicle that was being tracked before, due to obvious changes in the vehicle itself such as body structure distortion and cabin angle changes during the dumping operation, its characteristics are completely different from before, so it will be identified as a different situation during tracking judgment. Moreover, the difference in vehicle characteristics is particularly prominent during the dumping process, which will not be explained in detail here. Even after the dumping process is over, although the difference in vehicle characteristics has become relatively small, there are large changes in characteristics during the entire tracking process, which has a greater impact on the continuity of tracking. The second is the increase in slag accumulation.

[0055] S108, performing volume calculation strategies on other foreign bodies respectively to obtain other volume information and other volume sequences; It should be noted that the principle and process of this step are similar to those of step S106, and reference may be made to step S106, which will not be repeated here.

[0056] S109, selecting two other volume information from all other volume information and determining them as first other volume information and second other volume information; The first other volume information and the second other volume information are two arbitrary volume values ​​selected from all other volume information, and are used for subsequent comparison and judgment.

[0057] In some embodiments, two volume information are selected from the volume information of all other alien objects as the first other volume information and the second other volume information. The selection of these two volume information is the basis for subsequent judgment on whether other alien objects are related to the originally tracked alien object. By comparing and analyzing the volume information of the other alien objects with the originally tracked alien object, possible targets can be found.

[0058] S110, determining whether a difference between the sum of the first other volume information and the second other volume information and the original volume information is less than a difference threshold; The scenario for executing this step is to further determine whether these other alien objects are likely to be alien objects that have lost tracking before. In some embodiments, the first other volume information and the second other volume information are summed, and then the difference between the sum and the original volume information is calculated. The difference is then compared with a pre-set difference threshold. If the difference is less than the difference threshold, it means that from a volume perspective, the two other alien objects may be associated with the original tracked alien object, providing a basis for subsequent further confirmation; if the difference is greater than or equal to the difference threshold, it can be basically determined that they are not highly correlated, and the two other alien objects can be excluded.

[0059] S111, if the difference is less than the difference threshold, determining a change trend of other volume sequences corresponding to the first other volume information and the second other volume information; It should be noted that in actual use, those illegal vehicles involved in the indiscriminate dumping of rubble usually present a form of leaking tops, that is, vehicles without car tarpaulins or semi-enclosed vehicles. There are many reasons behind this phenomenon based on the characteristics of actual scenarios. In the specific illegal scenario of indiscriminate dumping of rubble, speed is often the primary consideration for violators. It usually takes 2-3 hours to dismantle and install car tarpaulins, which is obviously an unacceptable time cost for violators who are eager to complete the indiscriminate dumping of rubble quickly. In addition, some conventional vehicles that do not leak out of the top may have valuables placed in the car when unloading, or there may be "Chaoshan" situations and problems with lost items. However, for illegal vehicles that dump rubble, their purpose is simply to dump the rubble quickly, and they do not care whether the items in the car will be damaged or damp due to the lack of protection such as car tarpaulins, nor do they worry about the loss of items. Therefore, vehicles without car tarpaulins or semi-enclosed vehicles with rubble directly leaking out of the top have become their first choice for illegal behavior, in order to meet their needs to quickly and easily complete the indiscriminate dumping of rubble.

[0060] Therefore, in actual use, the situation where the sum of the volume of a fully enclosed vehicle and the dumped objects is greater than the original volume information will not be considered.

[0061] When the difference between the sum of the first other volume information and the second other volume information and the original volume information is calculated to be less than the difference threshold. Because in actual slag transportation and dumping and other related scenes, the initially tracked alien objects (such as vehicles transporting slag) have some changes in form later (such as the vehicle starts to dump slag, and the slag gradually accumulates to form a new object form, etc.), these changed forms are reflected as other alien objects (one may be the vehicle itself in the dumping process, and the other may be the dumped slag dump), and their volume information should have a certain logical connection with the original alien object (the transport vehicle and the slag as a whole). From the perspective of volume, the difference within a reasonable range (less than the difference threshold) is only a preliminary condition, and it is necessary to further analyze the trend of their volume changes over time to verify whether these other alien objects are derivatives of the original alien objects under specific behaviors (such as dumping slag). For example, if a vehicle is in the normal process of dumping rubble, the volume of the vehicle itself may show a specific change trend due to factors such as changes in the angle of the vehicle compartment (such as appearing to be gradually decreasing in size from certain viewing angles, etc.), while the volume of the dumped rubble will show a trend of gradually increasing as the dumping continues, proving that they are closely related to the original alien objects and are their manifestations in different stages and forms.

[0062] S112: Other alien objects whose other volume sequence change trend corresponding to the first other volume information or the second other volume information is decreasing are determined as alien objects and tracked, and the corresponding volume information modifies the original volume information.

[0063] Here, the alien object refers to the target object that was previously tracked but lost tracking later. Determining it as an alien object means determining that the other alien object corresponding to the other volume sequence is the same as the alien object that lost tracking before. Tracking is the operation of continuing to pay attention to it and recording information such as its position and motion state. The original volume information is the volume value of the alien object that was initially tracked calculated before. The corresponding volume information refers to the latest volume information of the other alien object currently determined as the original alien object. Changing the original volume information is to update the previous original volume information with the latest volume information to ensure the accuracy of the volume data of the alien object.

[0064] The scenario is to finally confirm and restore the tracking of the alien object that has lost tracking before. In some embodiments, when it is found that the other volume sequence corresponding to a certain other alien object shows a decreasing trend, combined with the previous judgment conditions such as the volume difference, it is comprehensively determined that the other alien object is the alien object that has lost tracking before. Then restart the tracking mechanism for it to obtain its position, movement status and other information in real time. At the same time, the current volume information of the confirmed alien object is used to update the original volume information recorded previously, because its volume may have changed during the period of loss of tracking. After the update, the subsequent analysis of it (such as volume change trend analysis, etc.) can continue to be carried out based on more accurate data, ensuring the consistency and accuracy of the entire monitoring process.

[0065] In some preferred embodiments, considering the need for vehicle information integrity and comprehensive behavior trajectory, a strategy is adopted to superimpose the features in this scene (obtained by step S105) with the features in other scenes. Through such a combination, a complete sequence of a vehicle can be generated, which can show the behavior pattern of the vehicle in detail on the one hand, and provide convenient conditions for finding the vehicle quickly and accurately on the other hand. Take the actual scene as an example. It may happen that a car is driving normally in a certain scene at first, and the license plate is still installed on the body, and there is no action to remove the license plate. But when it moves to another scene, it makes the illegal behavior of removing the license plate and then dumping the slag. In this case, only by associating and integrating the information related to the vehicle in each scene can the complete information about the vehicle be formed and its entire behavior process be completely restored.

[0066] In addition, in other preferred embodiments, in order to obtain more detailed vehicle-related information, monitoring information from other platforms is obtained, features are extracted using the same method, and are superimposed according to different scenarios to obtain a more complete sequence.

[0067] It can be seen that the monitoring screens of different scenes are obtained by using the image acquisition devices arranged in multiple scenes, and the foreign objects therein are judged, so as to lock the objects of interest for subsequent processing. Then, the foreign objects are divided into several first sub-regions, and these first sub-regions are clustered according to similarity and interval to obtain several second sub-regions. The vehicles transporting slag are divided into several regions with obvious characteristics, singleness and large areas, which is convenient for distinguishing the characteristic parts of the vehicles and slag, and improving the accuracy and efficiency of the overall judgment. Considering that the slag is generally above and has the most irregular shape, the second sub-region inside the upper region is placed in the algorithm warehouse in order from large to small according to the complexity of the shape to determine whether it has the pre-stored slag characteristics; while some of the parts of the car are located below, and the shapes are often relatively regular and simple, so the second sub-region inside the lower region is placed in the algorithm warehouse in order from small to large according to the complexity of the shape to determine whether it has the pre-stored automobile parts characteristics. In this way, the feature judgment can be fully based on the distribution characteristics and shape characteristics of the slag and automobile parts themselves, reducing the difficulty of calculation and resource consumption. Then, the foreign objects are tracked. During tracking, other image acquisition devices of the corresponding scene are awakened to collect several target images containing alien objects. These devices, which are distributed in different positions and have different shooting angles, can collect multi-view images that can present the state of alien objects in all directions, providing a data basis for subsequent volume calculation. Then, the volume calculation strategy is started to calculate the original volume information of the alien object in the actual space, and then the volume data of the alien object is periodically updated at time intervals to form a volume sequence of the volume of the alien object changing over time, so as to grasp the volume change of the alien object in real time and provide a key basis for judging its behavior state. When the alien object cannot be tracked, all other alien objects in the monitoring screen will be obtained, and the volume calculation strategy will be performed on them respectively to obtain other volume information and other volume sequences. After that, two suitable other volume information are selected from all other volume information, which are determined as the first other volume information and the second other volume information. By judging whether the difference between their sum and the original volume information is less than the difference threshold, the possible original alien objects are screened out. If the difference meets the requirements, the change trend of other corresponding volume sequences is further determined, and other foreign objects with a decreasing trend in volume sequence are determined as original foreign objects, and tracked, and the original volume information is changed with the corresponding volume information. In the actual scene of dumping slag, facing many problems such as the occlusion of the vehicle by the slag, the constant change of the vehicle's own shape with the dumping action, and the complex and difficult to distinguish vehicle features, it effectively makes up for the defects of the relevant tracking methods that are easy to lose the target and easy to misjudge. It can not only accurately lock the target for continuous tracking, but also reduce the consumption of computing resources through reasonable area division and feature judgment. After the target is lost, it can find the target again by volume-related judgment, so that the entire tracking process can be smoothly continued, and the success rate of tracking illegal vehicles that dump slag is increased.

[0068] In actual use, a lot of dust is generated when dumping the slag. The dust spreads around the vehicle and the slag, making the boundary between the vehicle and the slag unclear in the target image. Although they can be distinguished in fact, they are difficult to distinguish from the image visual perspective, which interferes with the recognition of other foreign objects; at the same time, the raised dust causes the volume of other foreign objects to change inappropriately, making it more difficult to identify them.

[0069] In this case, it is necessary to perform a tolerant operation on the period covered by steps S107 to S112 (allowing the time to be extended), and along with this tolerant change in time, the relevant requirements closely related thereto naturally also need to make corresponding adaptive adjustments. Therefore, in some preferred embodiments, step S109 is replaced by selecting two of the other volume information whose distances are less than a preset separation threshold from all the other volume information, and determining them as the first other volume information and the second other volume information; the preset separation threshold is positively correlated with the time elapsed from the loss of tracking of the alien object to the present moment.

[0070] The timing for executing this step is when the monitoring system loses the tracking of the alien object and needs to search for a target that may be related to the original alien object from other objects in the monitoring screen. In some embodiments, when the tracking of the alien object is lost, there will be multiple other objects in the screen, and each object has corresponding other volume information. In order to find clues that may be related to the original alien object from these information, it is necessary to select two appropriate information from all other volume information. Here, by comparing the distances between the other volume information, the two information whose distances are less than the preset separation threshold are determined as the first other volume information and the second other volume information. Moreover, the preset separation threshold is positively correlated with the time elapsed from the loss of tracking of the alien object to the present, because as time goes by, the alien object and the surrounding environment may undergo more changes, and the volume of the object may also fluctuate more. For example, in a short period of time, the difference in volume between the two objects may be small, and the preset separation threshold can be set relatively small; but after a long time, the state of the object may change greatly, and the allowable volume difference range should be expanded accordingly, so the preset separation threshold should increase with the increase in time.

[0071] In step S110, the difference threshold is positively correlated with the time elapsed from the loss of tracking of the alien object to the present; In some embodiments, when the monitoring system loses tracking of the alien object, multiple other objects will appear in the picture. It is necessary to compare the difference between the volume information of these other objects and the original volume information of the original alien object to determine whether they may be the original alien object or are closely related to the original alien object. The difference threshold is positively correlated with the time from the loss of tracking of the alien object to the present, because as time goes by, the alien object may undergo more changes, such as some slag drifting away with the air. Therefore, in a short period of time, the difference between the volume of other alien objects and the volume of the original alien object may be small, and the difference threshold can be set relatively small; but after a long time, the allowable volume difference range should be expanded accordingly to adapt to the greater changes that may occur in the alien object, so the difference threshold should increase with the increase of time.

[0072] It can be seen that, considering the influence of dust interference and the passage of time, for example, after the dust floats away, the original volume of the object may change greatly due to changes in the visual situation. As time goes by, the uncertainty increases. At this time, associating the difference threshold with the duration can better accommodate the uncertainty of volume changes caused by dust and time factors, avoid misjudgment due to overly strict volume thresholds, thereby increasing the probability of screening out the original alien object from many other alien objects, and creating favorable conditions for subsequent re-tracking. At the same time, two other volume information with a distance less than the preset separation threshold are selected from all other volume information, and are determined as the first other volume information and the second other volume information. The preset separation threshold is positively correlated with the time from the loss of tracking of the alien object to the present, taking into account the spatial distance factor that changes with time in the dust interference scene. In the case where dust causes the picture to be blurred and the object features are difficult to clearly distinguish, the vehicle may drive away over time, and the distance between the slag and the vehicle increases. By positively associating the preset separation threshold with the duration, the judgment standard for the distance can be dynamically adjusted according to time. As time goes by, the distance restriction range is reasonably relaxed so that the alien objects corresponding to the two other selected volume information are more consistent with the spatial changes of the original target in a dusty environment and after the vehicle may move, avoiding errors in distance judgment caused by visual illusions caused by dust and distance changes caused by the vehicle leaving, thereby more accurately screening out possible original alien objects and enhancing the accuracy of re-determining the target.

[0073] In the above embodiment, how to track the vehicles transporting the slag is achieved. However, in actual use, it is necessary to find out whether these vehicles are performing the dumping operation of the slag.

[0074] See also Figure 2 , Figure 2 is another flow chart of the integrated analysis method applicable to multi-scene dynamic images in an embodiment of the present application; Therefore, in some preferred embodiments, step S104 further includes: S201, generating a corresponding alarm strategy; Among them, the alarm strategy refers to a series of response rules and measures formulated for specific abnormal situations, which are used to issue alarms and take corresponding actions in a timely manner when an abnormality is detected.

[0075] In some embodiments, if a vehicle transporting slag is discovered, an alarm strategy is generated accordingly based on information such as the current time and the specific location.

[0076] S202, determining whether the movement rhythm data of the alien object conforms to a preset uniform rule; The preset uniform rule is a pre-set standard used to measure whether the movement rhythm of the alien object conforms to a certain uniform and stable pattern. For example, it stipulates that during normal driving of the vehicle, the speed fluctuation range cannot exceed a certain percentage, and the pause time cannot be too long.

[0077] In some embodiments, the monitoring system will collect the movement data of the foreign object (the vehicle transporting the slag) in real time, such as the location, speed and other information of the vehicle. Then, the movement rhythm data of the vehicle is calculated based on these data. These movement rhythm data are compared with the preset uniform rules to determine whether the movement of the vehicle conforms to the normal and uniform pattern. If the movement rhythm of the vehicle conforms to the preset uniform rules, it means that the driving state of the vehicle is relatively stable, the data has a certain reliability, and the original subsequent processing can be carried out; if it does not conform, there may be an abnormal situation, which requires further analysis.

[0078] In some specific embodiments, the speed and acceleration of the vehicle are calculated based on the change of the position information. The speed and acceleration data within a period of time are sorted to form a moving rhythm data sequence. Then, according to the preset uniform rule, the fluctuation range of the speed and the threshold of the acceleration are set. If all the data are within the rule range, it is determined that the preset uniform rule is met; otherwise, it is determined that it does not meet the rule.

[0079] If it meets the preset uniform rule, execute step S203; if it does not meet the preset uniform rule, execute step S206.

[0080] S203: If the preset uniform rule is met, determining whether the movement rhythm data is within the movement rhythm feature interval in the feature library of landfill dumping; Among them, the feature database of illegal dumping is a database that stores a large amount of feature data related to illegal dumping behavior. It contains various behavioral feature information of vehicles during illegal dumping, such as movement rhythm characteristics, stay time, location information, etc. The movement rhythm feature interval refers to a feature value range set for the vehicle movement rhythm in the feature database of illegal dumping behavior, which is used to indicate the possible range of changes in the vehicle movement rhythm when illegal dumping behavior occurs.

[0081] After initially judging that the vehicle's movement state is relatively normal, further in-depth analysis is conducted to determine whether there is a possibility of indiscriminate dumping. In some embodiments, when the monitoring system determines that the movement rhythm data of the foreign object meets the preset uniformity rule, these movement rhythm data will be compared with the movement rhythm feature interval in the indiscriminate dumping action feature library. If the movement rhythm data falls within the feature interval, it means that the movement rhythm of the vehicle has some typical characteristics of indiscriminate dumping behavior, and there is suspicion of indiscriminate dumping; if it is not within the interval, it means that the movement rhythm of the vehicle does not match the typical characteristics of indiscriminate dumping behavior, and the possibility of indiscriminate dumping is temporarily ruled out.

[0082] In some specific embodiments, relevant data of the moving rhythm feature interval is extracted from the feature library of illegal dumping, including the minimum and maximum values ​​of the speed, the minimum and maximum values ​​of the pause time, etc. The moving rhythm data of the foreign body collected in real time is sorted out to extract the corresponding key indicators such as speed and pause time. Then, these key indicators are compared with the boundary value of the feature interval. If all indicators are within the feature interval, it is determined that the moving rhythm data is within the moving rhythm feature interval; otherwise, it is determined that it is not.

[0083] If it is within the moving rhythm feature interval in the feature library of landfill dumping, execute step S204; if it is not within the moving rhythm feature interval in the feature library of landfill dumping, execute step S205.

[0084] S204: If the movement rhythm is within the feature interval of the illegal dumping action feature library, it is determined to be illegal dumping, and the alarm strategy is activated; S205: If the object is not within the moving rhythm feature interval in the feature library of landfill dumping, the process returns to the step of tracking the foreign object until a preset time is reached or there is no foreign object in the monitoring screen.

[0085] It can be seen that when it is judged to be a slag transport vehicle, an alarm strategy is generated in advance to facilitate the improvement of subsequent alarm efficiency and further obtain its movement rhythm data. By judging whether the movement rhythm data meets the preset uniformity rules, those cases where similar features appear accidentally but are not actually dumping slag are screened out. Even if it meets the uniformity rules, it is still necessary to continue to judge whether it is within the movement rhythm feature interval in the feature library of slag dumping actions. Only when it is confirmed layer by layer that it meets the requirements, the alarm strategy is activated. This makes the judgment of the behavior of dumping slag more accurate, effectively avoids the misjudgment caused by the similarity of some normal transportation behaviors with the characteristics of dumping slag, and reduces the manpower and time resources consumed by supervisors to verify the misjudgment.

[0086] S206. If the preset uniformity rule is not met, all behavior data of the foreign object from the time it appears on the monitoring screen to the current moment are collected, and the behavior data includes: driving trajectory, stop duration, speed change, and turning frequency; S207, clustering the behavior data with various preset normal behavior pattern data; In some embodiments, after obtaining the behavior data of the foreign body, the monitoring system will cluster it with various preset normal behavior pattern data. The system will calculate the similarity between the behavior data and each normal behavior pattern data, and classify the behavior data into the closest normal behavior pattern category according to the similarity. If the behavior data can be clustered by a certain normal behavior pattern data, it means that the behavior of the vehicle is generally in line with the normal transportation mode.

[0087] In some specific embodiments, various types of preset normal behavior pattern data are used as cluster centers, and the collected behavior data are used as samples to be classified. The distance (such as Euclidean distance) between each behavior data sample and each cluster center is calculated. Then, according to the distance, the behavior data sample is assigned to the normal behavior pattern category represented by the closest cluster center, which is not limited here.

[0088] S208, when the behavior data is clustered by any normal behavior pattern data, deleting the alarm strategy; In some embodiments, when the monitoring system determines through cluster analysis that the behavior data of the abnormal object matches a normal behavior pattern data, the current behavior of the vehicle is considered normal. In order to avoid unnecessary alarms, the system will automatically delete the previously generated alarm strategy.

[0089] S209: When the behavior data is not clustered by any normal behavior pattern data, the process returns to the step of tracking the alien object to obtain the movement rhythm data of the alien object until a preset time is reached or there is no alien object in the monitoring screen.

[0090] In some embodiments, when the monitoring system finds through cluster analysis that the behavior data of the alien object does not match all the preset normal behavior pattern data, it will be considered that the vehicle's behavior is abnormal. At this time, the system will restart tracking the alien object and continue to collect its movement rhythm data. During the tracking process, the system will constantly check whether the preset time has been reached or whether the alien object can still be seen in the monitoring screen. If the preset time is reached, it means that the vehicle has been tracked for a long enough time, and it is still unclear whether its behavior is illegal, and the tracking can be temporarily stopped; if there is no alien object in the monitoring screen, it means that the vehicle has left the monitoring range and cannot continue to be tracked, and the current tracking process will also be stopped.

[0091] It can be seen that when it is judged that the movement rhythm data does not meet the preset uniform rules, the multi-dimensional behavior data of the alien object from its appearance to the present is collected, and these data can fully reflect its behavioral characteristics. Then the behavior data is clustered with the preset normal behavior pattern data. Through this comparative clustering, it can be accurately judged whether its behavior is normal. If it is clustered by the normal pattern, it means that it may be a normal situation. Deleting the alarm strategy can avoid false alarms, reduce unnecessary interference, and improve the accuracy of the alarm; if it is not clustered, continue to track and obtain new data, and judge repeatedly until the conditions are met.

[0092] After step S209, the method further includes: S210, when the preset time is reached or there is no foreign object in the monitoring screen, and the behavior data is not clustered by any normal behavior pattern data; outputting the behavior data to the database to be determined; The database to be determined is a database specifically used to store behavioral data that has not yet been clearly classified into known normal behavioral patterns. It provides a centralized storage space for further analysis and research of these special behavioral data, which helps to discover new behavioral patterns.

[0093] In some embodiments, when cluster analysis shows that the behavior data does not match any preset normal behavior patterns, the monitoring system extracts the behavior data from the temporary storage location and stores it in the to-be-determined database according to certain formats and rules. The purpose of this is to retain these special behavior data so that subsequent personnel can analyze them in more detail and explore whether there are new normal behavior patterns.

[0094] S211. When the database to be determined is determined, a new normal behavior pattern is added, and the behavior data is determined as sub-data of the new normal behavior pattern.

[0095] Among them, the new normal behavior pattern refers to a behavior pattern with universal significance and regularity that is discovered through the analysis of special behavior data in addition to the original preset normal behavior pattern.

[0096] In some embodiments, after further verification and confirmation by subsequent staff, these behavioral data with similar characteristics are defined as a new normal behavior pattern. Then, the records originally belonging to these behavioral data are marked as sub-data of the new normal behavior pattern from the database to be determined, and updated to the corresponding normal behavior pattern database.

[0097] It can be seen that when the behavior data is not clustered and reaches the preset time or there is no foreign object in the monitoring screen, it will be output to the database to be determined. The data is retained for subsequent analysis and further confirmation. When the database to be determined is determined, a new normal behavior pattern is added and the behavior data is used as sub-data, which can continuously expand the scope of normal behavior patterns, making the system more comprehensive in its understanding of normal behavior, and thus more accurately distinguishing normal from abnormal behavior in subsequent judgments.

[0098] After step S213, the method further includes: S214, when the preset time is reached or there is no foreign object in the monitoring screen, and the movement rhythm data is not within the movement rhythm feature interval in the feature library of illegal dumping of landfills; deleting the alarm strategy.

[0099] In some embodiments, the monitoring system will continue to track the foreign object, obtain its movement rhythm data in real time and compare it with the movement rhythm feature interval in the feature library of landfill dumping. When the preset time is reached or the foreign object is not seen in the monitoring screen, if the movement rhythm data is not within the feature interval, it means that there is currently insufficient evidence to show that the foreign object has engaged in landfill dumping. At this time, it is necessary to execute the operation of deleting the alarm strategy to avoid unnecessary alarm interference.

[0100] It can be seen that if the movement rhythm data is not within the interval of the feature library of illegal dumping, the alarm strategy will be deleted after continuous tracking until the preset time or the condition of no foreign matter is met. This is to take into account that there may be special circumstances that cause the rhythm data to be temporarily abnormal but not illegal dumping. Deleting the alarm strategy can avoid false alarms caused by such circumstances, reduce invalid alarm information, and allow the monitoring system to focus on real illegal dumping.

[0101] The following introduces an exemplary integrated analysis system 400 applicable to multi-scene dynamic images provided in an embodiment of the present application. Figure 4 Schematic diagram of an exemplary hardware structure of an integrated analysis system 400 applicable to multi-scene dynamic images provided in an embodiment of the present application.

[0102] In some embodiments, the integrated analysis system 400 suitable for multi-scene dynamic images is a computer device or the integrated analysis system 400 suitable for multi-scene dynamic images includes a computer device. The computer device includes a processor, a memory and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with other external terminals or servers through a network connection. In some embodiments, the network interface can be a wired network interface, and in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, the method in the embodiment of the present application is implemented.

[0103] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0104] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0105] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0106] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0107] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0108] At present, a common technical means for supervising the transportation and dumping of slag is to rely on video surveillance by fixed cameras installed in key urban areas (such as construction site entrances and exits, along major transportation roads, etc.). The presence of indiscriminate dumping of slag is judged based on factors such as the vehicle's appearance, driving trajectory, and parking location. For example, if a vehicle is found to be parked near a non-designated slag disposal site, or if the vehicle's driving trajectory deviates from the prescribed transportation route and drives to some remote areas, it will be marked as a suspected violation.

[0109] In actual applications, since normal construction waste transportation operations themselves involve many complex scenarios and situations, some normal transportation behaviors exhibit characteristics that are extremely similar to those of indiscriminate dumping of construction waste, which can easily lead to misjudgments. Supervisors need to spend a lot of energy to verify these misjudgments one by one, wasting precious manpower and time resources.

Claims

1. An integrated analysis method suitable for multi-scene dynamic images, characterized in that: include: Acquire monitoring images of different scenes through image acquisition devices arranged in multiple scenes; Analyze the preset area of ​​the acquired monitoring screen to determine whether there is a foreign object in the preset area, wherein the foreign object refers to an object whose elements in the preset area differ from other areas of the monitoring screen by more than a preset threshold; The alien object is divided into a plurality of first sub-regions, and the first sub-regions are clustered according to similarity and interval to obtain a plurality of second sub-regions; wherein the higher the similarity or the lower the interval, the higher the probability of clustering; The foreign body is divided into an upper area and a lower area according to a height ratio threshold, and the second sub-areas in the upper area are placed into an algorithm warehouse in order of shape complexity from large to small to determine whether they have pre-stored slag features; Putting the second sub-regions inside the lower region into the algorithm warehouse in order of shape complexity from small to large, and judging whether they have pre-stored automobile part features; When any of the second sub-areas in the upper area has a pre-stored feature of slag, and any of the second sub-areas in the lower area has a pre-stored feature of automobile parts, tracking the foreign object; During tracking, other image acquisition devices of the corresponding scene are awakened to acquire several target images containing the alien object, the other image acquisition devices are arranged at different positions of the corresponding scene, have different shooting angles, and start a volume calculation strategy; the volume calculation strategy includes: calculating the original volume information of the alien object in the actual space according to the height, angle, focal length of the other image acquisition devices and the pixel size and position information occupied by the alien object in the target image; periodically updating the volume data of the alien object at time intervals to form a volume sequence of the volume of the alien object changing over time; In the case where the alien object cannot be tracked, obtaining all other alien objects in the monitoring screen; Performing volume calculation strategies on the other foreign objects respectively to obtain other volume information and other volume sequences; Selecting two pieces of the other volume information from all the other volume information to determine them as first other volume information and second other volume information; Determine whether a difference between a sum of the first other volume information and the second other volume information and the original volume information is less than a difference threshold; If it is less than the difference threshold, determining a change trend of other volume sequences corresponding to the first other volume information and the second other volume information; The other alien object whose other volume sequence change trend corresponding to the first other volume information or the second other volume information is decreasing is determined as the alien object and tracked, and the corresponding volume information modifies the original volume information.

2. The method according to claim 1, characterized in that The difference threshold is positively correlated with the time elapsed from the loss of tracking of the alien object to the present moment; The step of selecting two pieces of the other volume information from all the other volume information and determining them as the first other volume information and the second other volume information specifically includes: Selecting two pieces of the other volume information whose distances are less than a preset separation threshold from all the other volume information, and determining them as the first other volume information and the second other volume information; The preset separation threshold is positively correlated with the time elapsed from the loss of tracking of the alien object to the present moment.

3. The method according to claim 1, characterized in that: After the step of tracking the foreign object, the method further comprises: Generate corresponding alarm strategies; Determining whether the movement rhythm data of the alien object conforms to a preset uniform rule; If the preset uniform rule is met, determining whether the movement rhythm data is within the movement rhythm feature interval in the feature library of landfill dumping; If it is within the moving rhythm feature interval in the feature library of landfill dumping, it is determined to be landfill dumping and the alarm strategy is activated; If it is not within the moving rhythm feature interval in the feature library of landfill dumping, the process returns to the step of tracking the foreign object until a preset time is reached or there is no foreign object in the monitoring screen.

4. The method according to claim 3, characterized in that: After the step of determining whether the moving rhythm data conforms to a preset uniform rule, the method further comprises: If the preset uniform rule is not met, all behavior data of the foreign object from the time it appears on the monitoring screen to the current moment are collected, and the behavior data include: driving trajectory, stop duration, speed change, and turning frequency; Clustering the behavior data with various preset normal behavior pattern data; In the case where the behavior data is clustered by any of the normal behavior pattern data, deleting the alarm strategy; When the behavior data is not clustered by any of the normal behavior pattern data, the process returns to the step of tracking the alien object to obtain the movement rhythm data of the alien object until a preset time is reached or the monitoring screen does not contain the alien object.

5. The method according to claim 4, characterized in that In the case where the behavior data is not clustered by any of the normal behavior pattern data, the step of returning to the step of tracking the alien object to obtain the movement rhythm data of the alien object is returned until a preset time is reached or there is no alien object in the monitoring screen, and the method further includes: When the preset time is reached or there is no foreign object in the monitoring screen, the behavior data is not clustered by any normal behavior pattern data; Outputting the behavior data to a database to be determined; When the database to be determined is determined, a new normal behavior pattern is added, and the behavior data is determined as sub-data of the new normal behavior pattern.

6. The method according to claim 3, characterized in that If the object is not within the moving rhythm feature interval in the feature library of landfill dumping, the step of returning to the step of tracking the foreign object is continued until a preset time is reached or the monitoring screen does not contain the foreign object; When the preset time is reached or there is no foreign object in the monitoring screen, and the movement rhythm data is not within the movement rhythm feature interval in the feature library of illegal land dumping, the alarm strategy is deleted.

7. The method according to claim 1, characterized in that The step of placing the second sub-regions inside the upper region into the algorithm warehouse in order of shape complexity from large to small, and judging whether they have pre-stored muck features specifically includes: According to the image acquisition device corresponding to the second sub-area, a corresponding algorithm is selected from the algorithm warehouse; According to the algorithm warehouse, computing resources are allocated to determine whether there are pre-stored slag features.

8. An integrated analysis system suitable for multi-scene dynamic images, characterized in that: The integrated analysis system applicable to multi-scene dynamic images includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the integrated analysis system applicable to multi-scene dynamic images to execute the method described in any one of claims 1-7.

9. A computer program product comprising instructions, characterized in that When the computer program product runs on an integrated analysis system applicable to multi-scene dynamic images, the integrated analysis system applicable to multi-scene dynamic images executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an integrated analysis system applicable to multi-scene dynamic images, the integrated analysis system applicable to multi-scene dynamic images executes the method as described in any one of claims 1-7.

Images