A method for off-site operation analysis based on video data

Abstract

The present invention provides a method for analyzing off-site business operations based on video data, including: collecting violation images within a preset area through a preset monitoring device, transmitting the violation images to a preset NASNet network mechanism for training and learning to construct a corresponding first model; wherein, the NASNet network mechanism is a graphic classification architecture constructed based on a neural structure search framework; obtaining first weight information of the first model, transmitting the first weight information to a preset semantic segmentation network for distillation training to separate a number of second models; wherein, there are at least four second models; deploying an intelligent off-site business operation analysis mechanism through the first model and the second models, and identifying and classifying the violation situations of off-site business operations through the intelligent off-site business operation analysis mechanism.

Description

Technical Field

[0001] The present invention relates to the technical fields of intelligent recognition and urban governance, and particularly relates to a method for analyzing out-of-store business operations based on video data. Background Art

[0002] Currently, in the national urban environmental governance, there are often common problems such as out-of-store business operations, road occupation for business, street drying, garbage stacking, accumulated construction waste, illegal construction waste trucks, illegal parking, illegal billboards, random stacking of shared bicycles, and crowd gathering. Relevant departments often need to spend a lot of time and effort to check and control urban management violations. Conducting city-wide inspections not only consumes time and effort but also easily misses other areas, greatly wasting human resources.

[0003] This technical solution provides a precise monitoring, judgment, tracking, and recording of out-of-store business operations on existing monitoring devices, which not only consumes less cost but also improves the accuracy and timeliness of urban management work, and technology helps urban management. Summary of the Invention

[0004] The present invention provides a method for analyzing out-of-store business operations based on video data to solve the above problems.

[0005] The present invention provides a method for analyzing out-of-store business operations based on video data, which is characterized by including:

[0006] Collecting illegal images within a preset area through a preset monitoring device, and transmitting the illegal images to a preset NASNet network mechanism for training and learning to construct a corresponding first model; wherein,

[0007] The NASNet network mechanism is a graph classification architecture constructed based on a neural structure search framework;

[0008] Obtaining the first weight information of the first model, and transmitting the first weight information to a preset semantic segmentation network for distillation training to separate several second models; wherein,

[0009] There are at least four of the second models;

[0010] Deploying an intelligent out-of-store business operation analysis mechanism through the first model and the second models, and identifying and classifying the illegal situations of out-of-store business operations through the intelligent out-of-store business operation analysis mechanism.

[0011] As an embodiment of this technical solution, collecting illegal images within a preset area based on the preset monitoring device includes:

[0012] Collecting out-of-store business operation videos within a preset area through a preset monitoring device;

[0013] Based on the preset AI video analysis system, the image of interest is captured from the off-store business video and the input image is collected; wherein,

[0014] The AI ​​video analysis system is used to analyze and capture images containing illegal behaviors in the off-store business videos;

[0015] The image of interest capture is used to perform image recognition and image capture when a preset image of interest appears in a video;

[0016] Detect whether the input image has any violation phenomenon and determine the detection result; wherein,

[0017] When the detection result shows that the input image has a violation phenomenon, a corresponding violation image is determined; wherein,

[0018] The violations include one or more of occupying the road for business, drying clothes along the street, dumping garbage, accumulated debris, illegal dump trucks, illegal parking, illegal billboards and random stacking of shared bicycles.

[0019] As an embodiment of the present technical solution, the step of transmitting the illegal image to a preset NASNet network mechanism for training and learning to construct a corresponding first model includes:

[0020] Based on the gradient optimization method preset by the NASNet network mechanism, the illegal image is processed, and feature extraction is performed on the illegal image to determine the extracted features; wherein,

[0021] The gradient optimization method is used to continuously optimize the characteristic elements in the illegal image;

[0022] By extracting the features, a corresponding feature sample set is established, the feature sample set is classified, and the classification label is determined; wherein,

[0023] The feature sample set includes a training sample set, a verification sample set and a test sample set;

[0024] Based on a preset distributed system, distributed nodes are collected, and based on the classification labels, the illegal images are compressed into data packets, and the data packets are transmitted to the distributed nodes to establish spatial nodes;

[0025] Based on a preset normalized exponential function, the spatial nodes are processed to generate a corresponding label sample matrix, and at the same time, sample weight information of the label sample matrix is ​​obtained;

[0026] The corresponding first model is constructed through the sample weight information; wherein,

[0027] The first model is a model for classifying illegal images through a preset teacher classifier.

[0028] As an embodiment of the present technical solution, the obtaining of the first weight information of the first model includes:

[0029] Traverse the spatial nodes in the first model through preset traversal conditions and traversal rules to determine the traversed nodes, perform weighted calculation on the traversed nodes to determine the weighted nodes;

[0030] Based on the weighted nodes, optimize the structural weight and network weight of the first model to determine the optimized weight;

[0031] Select the optimal activation function and optimal spatial nodes in the optimized weight;

[0032] Construct an optimal sample matrix through the optimal activation function and optimal spatial nodes, and determine the first weight information of the optimal sample matrix.

[0033] As an embodiment of the present technical solution, the transmitting the first weight information to a preset semantic segmentation network for distillation training to construct a second model includes:

[0034] Perform sparse training on the first model through the convolutional network in the preset semantic segmentation network to generate first training data;

[0035] Obtain the first weight information of the first training data, transmit the first weight information to the preset semantic segmentation network for semantic cutting to determine the cutting weight information, update the first weight information through the cutting weight information, and record the update process;

[0036] Perform regular term induction and secondary sparsification on the update process of the first weight information until the corresponding model matrix generates the simplest matrix;

[0037] Generate a distillation trainer through the simplest matrix, and perform distillation training on the extracted features corresponding to the illegal images based on the distillation trainer to generate training data;

[0038] Train the feature sample set in the first model through the training data to construct a second model.

[0039] As an embodiment of the present technical solution, the deployment of the intelligent off-site operation analysis mechanism through the first model and the second model includes:

[0040] Obtain an illegal image, input the illegal image into the first model for preliminary classification to determine the preliminary classification result;

[0041] Based on the preliminary classification results, determine the corresponding classification labels, and through the classification labels, train and generate the corresponding teacher classifier;

[0042] Input the preliminary classification results into the teacher classifier for classification to determine the teacher classification results;

[0043] Input the teacher classification results into the second model for fine classification to refine the second classification results;

[0044] Collect the second classification labels corresponding to the second classification results, and through the second classification labels, generate a student classifier;

[0045] Import the student classifier into the second model to generate a target model, and through the first model and the target model, deploy an intelligent off-site business operation analysis mechanism.

[0046] As an embodiment of this technical solution, through the intelligent off-site business operation analysis mechanism, identify and classify the violations of off-site business operations, including:

[0047] Based on the preset constraint conditions and recognition rules in the intelligent off-site business operation analysis mechanism, quickly collect and identify the off-site business operation situations in the surveillance video to obtain violation recognition information; among them,

[0048] The preset constraint conditions at least include high-frequency violation areas, high-frequency violation times, and high-frequency violation types;

[0049] The violation recognition information at least includes violation recognition areas, violation recognition times, and violation recognition status data;

[0050] Classify the violation recognition information to obtain; among them,

[0051] The violation classification information includes violation types and violation frequency information; among them, the violation types include: known violation types and unknown violation types; among them,

[0052] The known violation types include: the first violation type, the second violation type, and the third violation type; among them,

[0053] The first violation type includes: off-site extended business type, occupying the road for selling type, public area drying type;

[0054] The second violation type includes: garbage illegally placed type, waste accumulation type, illegal publicity type;

[0055] The third violation type includes: illegal driving type, illegal parking type, electric vehicle occupying the road type, and personnel illegally occupying land and gathering type;

[0056] According to the preset level classification, classify different types of violation classification information to determine the violation classification and grading information of different violation degrees.

[0057] As an embodiment of this technical solution, the step of classifying different types of violation classification information according to the preset level classification to determine the violation classification and grading information of different violation degrees includes the following steps:

[0058] Step S10: Obtain the violation frequency information and violation time corresponding to the violation classification information by screening the data of the violation classification information; among them,

[0059] Step S11: Determine the violation information according to the violation frequency information and violation time.

[0060] Step S12: Retrieve the standard indicators in the preset compliance standard criteria, and compare and analyze the violation degree of the violation information through the standard indicators.

[0061] Step S13: Evaluate the violation degree to generate an evaluation result, and based on the evaluation result, classify different types of violation classification information to determine the violation classification and grading information of different violation degrees; among them,

[0062] The violation degrees include: serious violation degree, general violation degree, and minor violation degree.

[0063] As an embodiment of this technical solution, the rapid acquisition and recognition of the off-site business situation in the surveillance video to obtain violation recognition information includes:

[0064] Obtain an extended image by performing detection and extension processing on the detected violation images of off-site business in the surveillance video; among them,

[0065] The detection and extension processing at least includes detection range extension, detection time extension, and detection frequency extension;

[0066] The extended image is composed of several violation images;

[0067] Scan the extended image and perform violation recognition on the extended image to generate extended violation recognition information;

[0068] Obtain the corresponding violation classification information and violation level information by performing violation data analysis on the extended violation recognition information;

[0069] Generate an extended violation accuracy by calculating the difference between the violation classification information and violation level information and the violation information before extension respectively, and make a judgment; among them,

[0070] When the violation accuracy is within the preset threshold range, it is accurate identification;

[0071] When the violation accuracy is not within the preset threshold range, second extended detection is performed.

[0072] As an embodiment of this technical solution, the intelligent off-site operation analysis mechanism is used to identify and classify off-site operation violations, and the safety analysis further includes:

[0073] Based on the preset monitoring equipment, the off-site situation is detected to obtain real-time off-site information; among them,

[0074] The real-time off-site information includes: area information, time information, portrait information, and material information; among them,

[0075] The area information includes: violation area and normal area;

[0076] Through the real-time off-site information, safety analysis is performed on the violation area and the normal area respectively to obtain safety values and perform safety judgments; among them,

[0077] The safety values include: violation area safety value and normal area safety value;

[0078] The safety analysis includes: violation area safety analysis and normal area safety analysis; among them,

[0079] The violation area safety analysis extracts violation feature information and uses the preset violation comparison information to generate a violation area safety value; among them,

[0080] The violation feature information includes: violation classification information, violation level information, violation scenario data, violation time, and violation structure data;

[0081] The normal area safety analysis extracts suspicious feature information and uses the preset suspicious comparison information to generate a normal area safety value;

[0082] When the violation area safety value and the normal area safety value are both within the corresponding preset threshold ranges, it is a safe situation;

[0083] When the violation area safety value and the normal area safety value are not both within the corresponding preset threshold ranges, it is a potential hazard situation, and warning processing is performed.

[0084] The beneficial effects of the present invention are as follows: This article proposes to build multi-algorithm capabilities based on AI artificial intelligence technology. According to the rules of automatic judgment, it is divided into multi-task and multi-category situations. In order to improve the accuracy of automatic judgment, the judgment under multi-task and multi-category is further divided into smaller tasks. When performing AI analysis, according to the set rules, for the video or scene to be judged, this solution proposes a solution measure based on a large model driving a small model to perform video structuring applications.

[0085] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in the written specification and the accompanying drawings.

[0086] The technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0087] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention.

[0088] In the drawings:

[0089] Figure 1 is a method flow chart of an off-premises business analysis method based on video data in an embodiment of the present invention;

[0090] Figure 2 is a method flow chart of an off-premises business analysis method based on video data in an embodiment of the present invention. Detailed Embodiments

[0091] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0092] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it can be directly on the other component or indirectly on the other component. When a component is referred to as being "connected to" another component, it can be directly or indirectly connected to the other component.

[0093] It should be understood that the orientation or positional relationship indicated by terms such as "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0094] In addition, it should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The meaning of "a plurality" is two or more, unless otherwise specifically defined. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0095] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

[0096] Embodiment 1:

[0097] According to Figure 1 As shown, an off-premises business analysis method based on video data is provided in an embodiment of the present invention, which is characterized by including:

[0098] Collecting violation images within a preset area through a preset monitoring device, transmitting the violation images to a preset NASNet network mechanism for training and learning, and constructing a corresponding first model; wherein,

[0099] The NASNet network mechanism is a graphic classification architecture constructed based on a neural structure search framework;

[0100] Obtaining the first weight information of the first model, transmitting the first weight information to a preset semantic segmentation network for distillation training, and separating a number of second models; wherein,

[0101] There are at least four of the second models;

[0102] Deploy an intelligent off-site operation analysis mechanism through the first model and the second model, and identify and classify violations in off-site operations through the intelligent off-site operation analysis mechanism.

[0103] The working principle and beneficial effects of the above technical solution are as follows:

[0104] An embodiment of the present invention provides a method for analyzing off-site operations based on video data, which is used to collect violation images at a preset position based on a preset monitoring device, transmit the violation images to a preset NASNet network mechanism for training and learning, and construct a corresponding first model; the NASNet network mechanism is a graphic classification architecture constructed through a neural structure search framework; obtain the first weight information of the first model, transmit the first weight information to a preset semantic segmentation network for distillation training, and construct a second model; deploy an intelligent off-site operation analysis mechanism through the first model and the second model, and identify and classify violations in off-site operations through the intelligent off-site operation analysis mechanism. This article proposes to build multi-algorithm capabilities based on AI artificial intelligence technology, and divide them into multi-task and multi-category situations according to the rules of automatic judgment. In order to improve the accuracy of automatic judgment, the judgment under multi-task and multi-category is further divided into smaller tasks. When performing AI analysis, according to the set rules, for the video or scene to be judged, this solution proposes a solution measure based on the large model driving the small model to perform video structuring applications.

[0105] Embodiment 2:

[0106] According to Figure 2 As shown, this technical solution provides an embodiment. The collection of violation images in a preset area based on a preset monitoring device includes:

[0107] Collect off-site operation videos in a preset area through a preset monitoring device;

[0108] Based on a preset AI video judgment system, grab images of interest from the off-site operation videos and collect input images; where

[0109] The AI video judgment system is used to judge and capture images containing violation behaviors in off-site operation videos;

[0110] The image grabbing of interest is used to perform image recognition and capture when a preset image of interest appears in the video;

[0111] Detect whether there is a violation in the input image and determine the detection result; where

[0112] When the detection result is that there is a violation in the input image, determine the corresponding violation image; where

[0113] The described irregularities include one or more of occupying the road for business, drying along the street, garbage stacking, accumulated construction waste, illegal construction waste trucks, illegal parking, illegal billboards, and disorderly stacking of shared bicycles.

[0114] The working principle and beneficial effects of the above technical solution are as follows:

[0115] Based on a preset monitoring device, this technical solution collects irregularity images at preset positions, and through the preset monitoring device, collects videos of out-of-store operations within the monitoring range. Currently, most streets and alleys are equipped with monitoring devices to ensure the safety of citizens. By using the existing monitoring devices, videos of store operations are collected to reduce cost expenditures. Based on a preset AI video analysis system, it grabs images of interest from the videos of out-of-store operations, collects input images, and through the interest setting of the area of interest, automatically grabs the area of interest. The grabbing of images of interest is used to perform image recognition and image capture when preset images of interest appear in the video, improving work efficiency and avoiding setting up special staff for investigation. Through the AI video analysis system, it detects whether irregularities appear in the input images, and when irregularities appear in the input images, it determines that the irregularity images include one or more of occupying the road for business, drying along the street, garbage stacking, accumulated construction waste, illegal construction waste trucks, illegal parking, illegal billboards, and disorderly stacking of shared bicycles. These phenomena cause great problems for urban governance. Through automated intelligent recognition, it automatically selects the irregular images and uploads them to the control terminal of the staff for reminder.

[0116] Embodiment 3:

[0117] This technical solution provides an embodiment. The transmission of the irregularity images to a preset NASNet network mechanism for training and learning to construct a corresponding first model includes:

[0118] Based on the gradient optimization method preset by the NASNet network mechanism, process the irregularity images, extract features from the irregularity images, and determine the extracted features; where

[0119] The gradient optimization method is used to continuously optimize the feature elements in the irregularity images;

[0120] Through the extracted features, establish a corresponding feature sample set, classify the feature sample set, and determine the classification labels; where

[0121] The feature sample set includes a training sample set, a validation sample set, and a test sample set;

[0122] Based on a preset distributed system, collect distributed nodes, and based on the classification labels, compress the irregularity images into data packets and transmit the data packets to the distributed nodes to establish spatial nodes.

[0123] Process the spatial nodes based on a preset normalized exponential function to generate a corresponding label sample matrix. Meanwhile, obtain the sample weight information of the label sample matrix;

[0124] Construct a corresponding first model through the sample weight information; wherein,

[0125] The first model is a model for classifying illegal images through a preset teacher classifier.

[0126] The working principle and beneficial effects of the above technical solution are as follows:

[0127] This technical solution transmits illegal images to the NASNet network mechanism of a preset network for training and learning, and constructs a corresponding first model. The first model is a large model based on the NASNet network mechanism. Through preset label classification, the illegal image dataset is divided into three major categories. For example, the first illegal category is category A, the second illegal category is category B, and the third illegal category is category C, etc. Through the setting of illegal categories, based on the gradient optimization method preset by the NASNet network mechanism, illegal images can be processed, and feature extraction, sample set establishment, and spatial node establishment can be performed on illegal images; the sample set includes a training sample set, a validation sample set, and a test sample set. Through the preset Softmax function and spatial nodes of the NASNet network mechanism, a sample matrix is constructed, and the sample weight information of the sample matrix is obtained; through the sample weight information, a corresponding first model is constructed. The graphic classification dataset is very large. Through the NASNet network mechanism, using data-driven and intelligent methods, an optimized model is constructed through complex convolutions to improve the training efficiency of illegal images.

[0128] Example 4:

[0129] This technical solution provides an example. The obtaining of the first weight information of the first model includes:

[0130] Traverse the spatial nodes in the first model according to a preset traversal condition and traversal rule to determine the traversed nodes, and perform weighted calculation on the traversed nodes to determine the weighted nodes;

[0131]

[0132] represents the weighted node, x i represents the i-th spatial node that has been traversed, i = 1, 2,..., n, and n represents the total number of spatial nodes, represents the spatial node x traversed under the traversal condition S and traversal rule R i , Represents the weight of the spatial node x traversed under the traversal condition S and the traversal rule R i , Represents the weight of the spatial node x traversed under the traversal condition S and the traversal rule R i-1 , σ represents the influence factor in the first model, j = 1, 2, …, m, m represents the total number of traversed nodes, g j Represents the iteration value of the j-th traversed node, M represents the average quality of the spatial nodes in the first model, Q represents the preset spatial index evaluation quality in the first model, h k Represents the quality entropy weight of the preset evaluation index for each spatial node in the first model, k = 1, 2, …, K, K represents the total number of evaluation indexes;

[0133] Based on the weighted nodes, optimize the structural weight and network weight of the first model to determine the optimized weight;

[0134]

[0135] Among them, G is the identifier of the first model, OP(G) represents the optimized weight regarding the first model, F represents the optimized scheduling function regarding the weighted nodes, f 1 Represents the structural weight, f 2 Represents the network weight, Represents the preset optimized weight of the first model, Represents the identifier of the ideal first model, s 1 <s 2 , and s 1 , s 2 ∈U, U represents the spatial range of the weighted nodes;

[0136] Select the optimal activation function and the optimal spatial node in the optimized weight;

[0137]

[0138] Among them, ε represents the preset difference threshold, max(x i ) represents the optimal spatial node, and the optimal activation function is obtained through the optimal spatial node;

[0139] Construct the optimal sample matrix through the optimal activation function and the optimal spatial node, and determine the first weight information of the optimal sample matrix.

[0140] The working principle and beneficial effects of the above technical solution are as follows:

[0141] When obtaining the first weight information of the first model to reduce classification errors during the training of the classifier of the training model, traverse the spatial nodes in the first model according to the preset traversal conditions and traversal rules to determine the traversed nodes, and perform weighted calculations on the traversed nodes to determine the weighted nodes. During the traversal process, since influence values and iteration values will be generated in the training space, and at the same time, in order to evaluate the importance of spatial nodes, it is necessary to perform weighted calculations on the nodes. Based on the weighted nodes, optimize the structural weights and network weights of the first model to determine the optimized weight OP(G); the optimized weight is converged in the structural weight f 1 and the network weight f2. The smaller the structural weight, the lighter the training of the classifier. The larger the network weight, the more category labels for classification and the more accurate the classification. Select the optimal activation function and the optimal spatial node max(x i ) in the optimized weight. Through the quality evaluation of the spatial nodes, select the optimized weight to reduce the error of model information and provide the original data for accurately training the classifier; through the optimal activation function and the optimal spatial node, construct the optimal sample matrix and determine the first weight information of the optimal sample matrix. Provide more optimized first weight parameters of the first model, which helps to simplify the model parameters in the second model, reduce the training time of searching, and improve the model training efficiency.

[0142] Example 5:

[0143] This technical solution provides an example. Transmitting the first weight information to a preset semantic segmentation network for distillation training to construct a second model includes:

[0144] Perform sparse training on the first model through the convolutional network in the preset semantic segmentation network to generate first training data;

[0145] Obtain the first weight information of the first training data, transmit the first weight information to the preset semantic segmentation network for semantic cutting to determine the cutting weight information, and update the first weight information through the cutting weight information, and record the update process;

[0146] Perform regular term induction and secondary sparsification on the update process of the first weight information until the corresponding model matrix generates the simplest matrix;

[0147] Generate a distillation trainer through the simplest matrix, and based on the distillation trainer, perform distillation training on the extracted features corresponding to the illegal images to generate training data;

[0148] Train the feature sample set in the first model through the training data to construct a second model.

[0149] The working principle and beneficial effects of the above technical solution are as follows:

[0150] In this technical solution, through the convolutional network in the preset semantic segmentation network, sparse training is performed on the first model to determine the first training data; the first weight information of the first training data is obtained, the first weight information is updated, and the update process is determined; regularization term induction and secondary sparsification are performed on the update process of the first weight information until the corresponding model matrix generates the simplest matrix; through the simplest matrix, a distillation trainer is generated, and distillation training is performed based on the distillation trainer to determine the training result. Based on the training result, a second model is constructed, and the first weight information is transmitted to the preset semantic segmentation network for distillation training to construct the second model. Through semantic segmentation, the weight information trained by the first model is refined, and the first model is migrated to the lightweight and small second model. Through migration, the classification of graphics can be carried out in multiple processes, and at the same time, the classification fineness of graphics can be improved, different graphics in different situations can be classified differently, the work efficiency can be improved, and the huge time cost of the NASNet network mechanism is avoided. Only by training the model once through the NASNet network for the first time, the migration of multiple lightweight models can be satisfied.

[0151] Embodiment 6:

[0152] This technical solution provides an embodiment. By using the first model and the second model, an intelligent off-site operation analysis mechanism is deployed, including:

[0153] Obtain a violation image, input the violation image into the first model for preliminary classification to determine the preliminary classification result;

[0154] Based on the preliminary classification result, determine the corresponding classification label, and through the classification label, train and generate the corresponding teacher classifier;

[0155] Input the preliminary classification result into the teacher classifier for classification to determine the teacher classification result;

[0156] Input the teacher classification result into the second model for fine classification to refine the second classification result;

[0157] Collect the second classification label corresponding to the second classification result, and through the second classification label, generate a student classifier;

[0158] Import the student classifier into the second model to generate a target model. Through the first model and the target model, an intelligent off-site operation analysis mechanism is deployed.

[0159] The working principle and beneficial effects of the above technical solution are as follows:

[0160] Through the first model and the second model, this technical solution deploys an intelligent off-site business operation analysis mechanism, including: obtaining a violation image, inputting the violation image into the first model for preliminary classification to determine the preliminary classification result; determining the corresponding preliminary classification label based on the preliminary classification result, and generating a teacher classifier through the preliminary classification label; inputting the preliminary classification result into the second model for fine classification through the teacher classifier to determine the second classification result; determining the corresponding second classification label based on the second classification result, and generating a student classifier through the second classification label; importing the student classifier into the second model to determine the target model, and deploying the intelligent off-site business operation analysis mechanism through the first model and the target model. By constructing the first model and the second model, the intelligent off-site business operation analysis mechanism is deployed, so as to automatically conduct intelligent analysis on the off-site business operation situation, reduce the workload of urban management staff, improve work efficiency, and contribute to urban governance.

[0161] Embodiment 7:

[0162] This technical solution provides an embodiment, which, through the intelligent off-site business operation analysis mechanism, identifies and classifies the violations of off-site business operations, including:

[0163] Based on the preset constraint conditions and recognition rules in the intelligent off-site business operation analysis mechanism, quickly collect and identify the off-site business operation situation in the surveillance video to obtain violation recognition information; among them,

[0164] The preset constraint conditions at least include high-frequency violation areas, high-frequency violation times, and high-frequency violation types;

[0165] The violation recognition information at least includes violation recognition areas, violation recognition times, and violation recognition status data;

[0166] Classify the violation recognition information to obtain,...; among them,

[0167] The violation classification information includes violation types and violation frequency information; among them, the violation types include: known violation types and unknown violation types; among them,

[0168] The known violation types include: the first violation type, the second violation type, and the third violation type; among them,

[0169] The first violation type includes: off-site extended business types, occupying the road for selling types, and public site drying types;

[0170] The second violation type includes: garbage violation placement types, waste accumulation types, and violation publicity types;

[0171] The third type of violation includes: types of illegal driving, types of illegal parking, types of electric vehicles occupying the road, and types of people illegally occupying land and gathering;

[0172] According to the preset level classification, classify the violation classification information of different types, and determine the violation classification and grading information of different violation degrees.

[0173] The working principle and beneficial effects of the above technical solution are as follows:

[0174] This technical solution uses an intelligent off-site operation analysis mechanism to identify and classify the violations of off-site operations. According to the preset training conditions, quickly collect the off-site operation situations to obtain violation identification information; the preset training conditions include: high-frequency violation areas, high-frequency violation times, high-frequency violation types; the violation identification information includes: violation areas, violation times, violation data; by classifying the violation identification information, obtain violation classification information and perform grading processing; the violation classification information includes: violation types, violation frequency information; the violation types include: known violation types, unknown violation types; the known violation types include: the first type of violation, the second type of violation, the third type of violation; the first type of violation includes: types of off-site extended operations, types of occupying the road for selling, types of drying in public places; the second type of violation includes: types of illegally placing garbage, types of waste accumulation, types of illegal publicity; the third type of violation includes: types of illegal driving, types of illegal parking, types of electric vehicles occupying the road, and types of people illegally occupying land and gathering; through the intelligent classification of different violation phenomena, improve the accurate identification of off-site operations.

[0175] Example 8:

[0176] This technical solution provides an example. The steps of classifying the violation classification information of different types according to the preset level classification and determining the violation classification and grading information of different violation degrees are as follows:

[0177] Step S10: Through data screening of the violation classification information, obtain the violation frequency information and violation time corresponding to the violation classification information; among them,

[0178] Step S11: Determine the violation information according to the violation frequency information and violation time;

[0179] Step S12: Retrieve the standard indicators in the preset compliance standard criteria, and through the standard indicators, compare and analyze the violation degree of the violation information;

[0180] Step S13: Evaluate the violation degree to generate an evaluation result. Based on the evaluation result, classify the violation classification information of different types and determine the violation classification and grading information of different violation degrees; among them,

[0181] The degrees of violation include: severe violation degree, general violation degree, and minor violation degree.

[0182] The working principle and beneficial effects of the above technical solution are as follows:

[0183] The hierarchical processing of this technical solution includes data screening of the violation classification information to obtain screening data; among them, the screening data includes: violation frequency information, violation time; the violation frequency information includes: the number of violations within a preset time, the rate of change of frequency within a preset time; numerical comparison and analysis are performed based on the screening data and the violation comparison data in the preset violation database to generate a violation comparison value; the numerical comparison and analysis include: average value comparison analysis, median value comparison analysis, mode comparison analysis; through the violation comparison value, the violation classification information is judged for violation grading; the violation grading judgment is to substitute the violation comparison value into the preset violation comparison table, find the corresponding violation interval, and obtain the violation level; the violation levels include: severe violation, general violation, minor violation, so as to meet the refinement of the violation information and improve the accuracy of the student classifier.

[0184] Embodiment 9:

[0185] This technical solution provides an embodiment for quickly collecting and identifying the situation of off-site business in the monitoring video to obtain violation identification information, including:

[0186] By performing detection and extension processing on the detected off-site business violation images in the monitoring video, an extended image is obtained; among them,

[0187] The detection and extension processing at least includes detection range extension, detection time extension, and detection frequency extension;

[0188] The extended image is composed of several violation images;

[0189] Scan the extended image and perform violation identification on the extended image to generate extended violation identification information;

[0190] By performing violation data analysis on the extended violation identification information, the corresponding violation classification information and violation level information are obtained;

[0191] By calculating the difference between the violation classification information and the violation level information and the violation information before extension respectively, an extended violation accuracy is generated and judged; among them,

[0192] When the violation accuracy is within the preset threshold range, it is accurate identification;

[0193] When the violation accuracy is not within the preset threshold range, a second extended detection is performed.

[0194] Through the intelligent off-site operation analysis mechanism, this technical solution identifies and classifies the violations of off-site operations, and also includes accuracy detection: through the preset detection expansion processing of the detected violation images, an extended image is obtained, and the extended image is subjected to violation identification to generate extended identification information; the detection expansion processing includes: detection scope expansion, detection time expansion, and detection frequency expansion; through the violation data analysis of the extended identification information, extended violation information is generated; the extended violation information includes: violation classification information and violation level information; by calculating the difference between the extended violation information and the violation information before extension, the extended violation accuracy is generated and judged; when the violation accuracy is within the preset threshold range, it is accurate identification; when the violation accuracy is not within the preset threshold range, a second extended detection is performed, so as to accurately expand different violation phenomena, optimize the off-site operation analysis mechanism, and improve the accuracy of off-site operation analysis.

[0195] Example 10:

[0196] This technical solution provides an example, which, through the intelligent off-site operation analysis mechanism, identifies and classifies the violations of off-site operations, and also includes safety analysis including:

[0197] Based on the preset monitoring equipment, the off-site situation is detected to obtain real-time off-site information; among them,

[0198] The real-time off-site information includes: area information, time information, portrait information, and material information; among them,

[0199] The area information includes: violation area and normal area;

[0200] Through the real-time off-site information, the safety analysis of the violation area and the normal area is respectively carried out to obtain safety values and perform safety judgments; among them,

[0201] The safety values include: violation area safety value and normal area safety value;

[0202] The safety analysis includes: violation area safety analysis and normal area safety analysis; among them,

[0203] The violation area safety analysis extracts violation feature information and uses the preset violation comparison information to generate a violation area safety value; among them,

[0204] The violation feature information includes: violation classification information, violation level information, violation scenario data, violation time, and violation structure data;

[0205] The normal area safety analysis extracts suspicious feature information and uses the preset suspicious comparison information to generate a normal area safety value;

[0206] When the safety values of the violation area and the normal area are both within the corresponding preset threshold ranges, it is a safe situation;

[0207] When the safety values of the violation area and the normal area are not both within the corresponding preset threshold ranges, it is a potential hazard situation, and early warning processing is carried out.

[0208] The working principle and beneficial effects of the above technical solution are as follows:

[0209] This technical solution uses an intelligent off-site operation analysis mechanism to identify and classify off-site operation violations, and also includes safety analysis: detecting off-site conditions based on preset monitoring devices to obtain real-time off-site information; the real-time off-site information includes: area information, time information, portrait information, and material information; the area information includes: violation area and normal area; performing safety analysis on the violation area and the normal area respectively through the real-time off-site information to obtain safety values and perform safety judgments; the safety values include: safety value of the violation area and safety value of the normal area; the safety analysis includes: safety analysis of the violation area and safety analysis of the normal area; the safety analysis of the violation area generates a safety value of the violation area by extracting violation feature information and using preset violation comparison information; among them, the violation feature information includes: violation classification information, violation level information, violation scenario data, violation time, and violation structure data; the safety analysis of the normal area generates a safety value of the normal area by extracting suspicious feature information and using preset suspicious comparison information; when the safety values of the violation area and the normal area are both within the corresponding preset threshold ranges, it is a safe situation; when the safety values of the violation area and the normal area are not both within the corresponding preset threshold ranges, it is a potential hazard situation, and early warning processing is carried out. By classifying the violations and areas of the violation area, the monitored area is focused, the detection of violation information is improved, and the burden on staff is reduced.

[0210] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0211] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to produce a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0212] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0214] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A method for analyzing off - store business based on video data, characterized in that, it includes: Collecting violation images within a preset area through a preset monitoring device, transmitting the violation images to a preset NASNet network mechanism for training and learning, and constructing a corresponding first model; where, The NASNet network mechanism is a graphic classification architecture constructed based on a neural structure search framework; Obtaining the first weight information of the first model, transmitting the first weight information to a preset semantic segmentation network for distillation training, and separating several second models; where, There are at least four of the second models; Deploying an intelligent off - store business analysis mechanism through the first model and the second models, and identifying and classifying the violation situations of off - store business through the intelligent off - store business analysis mechanism; The step of transmitting the violation images to a preset NASNet network mechanism for training and learning to construct a corresponding first model includes: Processing the violation images based on a preset gradient optimization method of the NASNet network mechanism, extracting features from the violation images, and determining the extracted features; where, The gradient optimization method is used to continuously optimize the feature elements in the violation images; Establishing a corresponding feature sample set through the extracted features, classifying the feature sample set, and determining classification labels; where, The feature sample set includes a training sample set, a validation sample set, and a test sample set; Collecting distributed nodes based on a preset distributed system, compressing the violation images into data packets based on the classification labels, and transmitting the data packets to the distributed nodes to establish spatial nodes; Processing the spatial nodes based on a preset normalization exponential function to generate a corresponding label sample matrix, and at the same time, obtaining the sample weight information of the label sample matrix; Constructing a corresponding first model through the sample weight information; where, The first model is a model for classifying violation images through a preset teacher classifier; The step of obtaining the first weight information of the first model includes: Traversing the spatial nodes in the first model through a preset traversal condition and traversal rule to determine traversal nodes, performing weighted calculations on the traversal nodes to determine weighted nodes; Optimizing the structural weight and network weight of the first model based on the weighted nodes to determine the optimized weight; Selecting the optimal activation function and the optimal spatial node in the optimized weight; Constructing an optimal sample matrix through the optimal activation function and the optimal spatial node, and determining the first weight information of the optimal sample matrix; The step of transmitting the first weight information to a preset semantic segmentation network for distillation training to construct a second model includes: Performing sparsification training on the first model through a convolutional network in the preset semantic segmentation network to generate first training data; Obtaining the first weight information of the first training data, transmitting the first weight information to the preset semantic segmentation network for semantic cutting to determine the cutting weight information, updating the first weight information through the cutting weight information, and recording the update process; Regular term induction and quadratic sparsification are performed on the update process of the first weight information until the corresponding model matrix generates the simplest matrix; Through the simplest matrix, a distillation trainer is generated. Based on the distillation trainer, distillation training is performed on the extracted features corresponding to the illegal images to generate training data; Through the training data, the feature sample set in the first model is trained to construct the second model.

2. A method for off-premises business operation analysis based on video data according to claim 1, characterized in that Based on the preset monitoring device, illegal images within a preset area are collected, including: Collect off-premises business operation videos within a preset area through a preset monitoring device; Based on a preset AI video research and judgment system, grab images of interest from the off-premises business operation videos to collect input images; wherein, The AI video research and judgment system is used to research and judge and capture images containing illegal behaviors in the off-premises business operation videos; The image grabbing of interest is used to perform image recognition and capture when a preset image of interest appears in the video; Detect whether the input image has illegal phenomena and determine the detection result; wherein, When the detection result is that the input image has illegal phenomena, determine the corresponding illegal image; wherein, The illegal phenomena include one or more of occupying the road for business operation, drying in the street, garbage stacking, accumulated construction waste, illegal construction waste trucks, illegal parking, illegal billboards, and random stacking of shared bicycles.

3. A method for off-premises business operation analysis based on video data according to claim 1, characterized in that Deploying an intelligent off-premises business operation analysis mechanism through the first model and the second model, including: Obtain illegal images, input the illegal images into the first model for preliminary classification, and determine the preliminary classification result; Determine the corresponding classification label through the preliminary classification result, and train and generate the corresponding teacher classifier through the classification label; Input the preliminary classification result into the teacher classifier for classification to determine the teacher classification result; Input the teacher classification result into the second model for fine classification to refine the second classification result; Collect the second classification label corresponding to the second classification result, and generate a student classifier through the second classification label; Import the student classifier into the second model to generate a target model, and deploy an intelligent off-premises business operation analysis mechanism through the first model and the target model.

4. A method for off-premises business operation analysis based on video data according to claim 1, characterized in that Identifying and classifying the illegal situations of off-premises business operations through the intelligent off-premises business operation analysis mechanism, including: Based on the preset constraint conditions and recognition rules in the intelligent off-premises business operation analysis mechanism, quickly collect and recognize the off-premises business operation situations in the monitoring video to obtain illegal recognition information; wherein, The preset constraint conditions at least include high-frequency illegal areas, high-frequency illegal times, and high-frequency illegal types; The illegal recognition information at least includes illegal recognition areas, illegal recognition times, and illegal recognition status data; Classify the illegal recognition information to obtain illegal classification information; wherein, The illegal classification information includes illegal types and illegal frequency information; among them, the illegal types include: known illegal types and unknown illegal types; among them, the known illegal types include: the first illegal type, the second illegal type, and the third illegal type; among them, the first illegal type includes: off-site extended business type, occupying the road for selling type, and drying in public places type; the second illegal type includes: illegal garbage placement type, waste accumulation type, and illegal publicity type; the third illegal type includes: illegal driving type, illegal parking type, electric vehicle occupying the road type, and personnel illegally occupying land and gathering type; According to the preset level classification, classify different types of illegal classification information to determine the illegal classification and grading information of different illegal degrees.

5. A method for analyzing off-site business based on video data as described in claim 4, characterized in that, the step of classifying different types of illegal classification information according to the preset level classification to determine the illegal classification and grading information of different illegal degrees includes the following steps: Step S10: Obtain the illegal frequency information and illegal time corresponding to the illegal classification information by screening the data of the illegal classification information; Step S11: Determine the illegal information according to the illegal frequency information and illegal time; Step S12: Retrieve the standard indicators in the preset compliance standard criteria, and compare and analyze the illegal degree of the illegal information through the standard indicators; Step S13: Evaluate the illegal degree to generate an evaluation result, and based on the evaluation result, classify different types of illegal classification information to determine the illegal classification and grading information of different illegal degrees; among them, the illegal degrees include: serious illegal degree, general illegal degree, and minor illegal degree.

6. A method for analyzing off-site business based on video data as described in claim 4, characterized in that, the step of quickly collecting and identifying the off-site business situation in the surveillance video to obtain illegal identification information includes: Obtain an extended image by performing detection and extension processing on the illegal images of off-site business detected in the surveillance video; among them, the detection and extension processing at least includes detection range extension, detection time extension, and detection frequency extension; the extended image is composed of several illegal images; Scan the extended image and perform illegal identification on the extended image to generate extended illegal identification information; Obtain the corresponding illegal classification information and illegal level information by performing illegal data analysis on the extended illegal identification information; Generate an extended illegal accuracy by calculating the difference between the illegal classification information and the illegal level information and the illegal information before extension respectively, and make a judgment; among them, When the illegal accuracy is within the preset threshold range, it is accurate identification; When the illegal accuracy is not within the preset threshold range, perform a second extended detection.

7. A method for analyzing off-site business based on video data as described in claim 1, characterized in that, the step of identifying and classifying the illegal situation of off-site business through the intelligent off-site business analysis mechanism further includes safety analysis including: Detect the situation outside the store based on a preset monitoring device to obtain real-time information outside the store; among them, The real-time information outside the store includes: area information, time information, and material information; among them, The area information includes: violation area, normal area; Perform safety analysis on the violation area and the normal area respectively through the real-time information outside the store, obtain safety values, and perform safety judgments; among them, The safety values include: violation area safety value, normal area safety value; The safety analysis includes: violation area safety analysis, normal area safety analysis; among them, The violation area safety analysis generates a violation area safety value by extracting violation feature information and using preset violation comparison information; among them, The violation feature information includes: violation classification information, violation level information, violation scenario data, violation time, violation structure data; The normal area safety analysis generates a normal area safety value by extracting suspicious feature information and using preset suspicious comparison information; When the violation area safety value and the normal area safety value are both within the corresponding preset threshold ranges, it is a safe situation; When the violation area safety value and the normal area safety value are not both within the corresponding preset threshold ranges, it is a potential hazard situation, and early warning processing is performed.

Images