Artificial intelligence-based monitoring video target detection method and system

Abstract

The application belongs to the technical field of artificial intelligence, and provides a monitoring video target detection method and system based on artificial intelligence, in order to solve the problem of low precision and efficiency of slag transportation supervision technical means in traditional technology, the method collects monitoring videos of different angles based on a multi-camera array, automatically detects the slag truck in the monitoring video, constructs an information feature matrix of the slag truck, and then detects the target of the abnormal slag truck in the monitoring video according to the information feature matrix, which can improve the supervision accuracy and efficiency of the slag transportation, so as to realize the precision and efficiency of the slag transportation supervision through the intelligent slag truck monitoring system based on artificial intelligence, create an efficient and accurate slag transportation supervision mode, improve the digital level of the slag transportation supervision, and improve the level of smart city and digital governance.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, smart cities and digital governance technologies, and in particular to a method and system for target detection in surveillance videos based on artificial intelligence. Background Technology

[0002] Construction waste is a mixed waste generated from construction projects, mainly including but not limited to excavated soil, construction waste, decoration waste, and construction sludge. Construction waste trucks are trucks specifically used to transport construction waste and construction waste. Construction waste is generally transported from the waste-generating site to the disposal site in accordance with relevant requirements, and then processed accordingly at the disposal site. The transportation of construction waste is generally carried out by responsible entities such as the construction unit (generating party) and professional transportation companies (carriers). In order to avoid problems such as overloading, dust pollution, waste spillage, illegal dumping, and "black construction waste trucks" that may occur during the transportation of construction waste, the corresponding responsible entities need to supervise the transportation of construction waste.

[0003] In traditional technologies, the supervision of construction waste trucks is generally based on manual inspections or by capturing surveillance videos at construction site vehicle exits, road checkpoints, and disposal site entrances.

[0004] However, the inventors discovered that in traditional technologies, monitoring dump trucks by capturing images from surveillance videos usually involves relevant personnel reviewing the videos to identify problems, or using the captured images as evidence after a problem occurs. Neither of these methods can detect problems in the transportation of construction waste in a timely and automatic manner.

[0005] Therefore, improving the accuracy and efficiency of technical means for supervising the transportation of construction waste has become an urgent problem to be solved in the field of smart cities and digital governance. Summary of the Invention

[0006] The technical problem solved by this invention is to address the issue of low accuracy and efficiency of technical means for supervising the transportation of construction waste.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a target detection method for surveillance videos based on artificial intelligence, comprising: acquiring exit surveillance videos of dump trucks leaving the site based on a preset first camera array, wherein the exit surveillance videos include a main exit surveillance video; detecting whether the main exit surveillance video contains a dump truck based on a pre-trained dump truck detection model; if the detection is positive, identifying the vehicle identification and model type of the dump truck, and constructing an initial information feature matrix of the dump truck based on the exit surveillance video and a preset feature matrix construction method; determining a preset standard information feature matrix corresponding to the model type; judging whether the initial information feature matrix is ​​similar to the preset standard information feature matrix based on a preset matrix similarity judgment method; if the judgment is positive, determining that the target dump truck corresponding to the vehicle identification is not abnormal.

[0008] This invention also provides an artificial intelligence-based surveillance video target detection system, comprising: a first acquisition module, configured to acquire exit monitoring video of a dump truck leaving the site based on a preset first camera array, the exit monitoring video including a main exit monitoring video; a first detection module, configured to detect whether the main exit monitoring video contains a dump truck based on a pre-trained dump truck detection model; a first identification module, configured to identify the vehicle identification and model type of the dump truck if the above detection is positive, and construct an initial information feature matrix of the dump truck based on the exit monitoring video and a preset feature matrix construction method; a first determination module, configured to determine a preset standard information feature matrix corresponding to the model type; a first judgment module, configured to determine whether the initial information feature matrix is ​​similar to the preset standard information feature matrix based on a preset matrix similarity judgment method; and a first determination module, configured to determine that the target dump truck corresponding to the vehicle identification is not abnormal if the above determination is positive.

[0009] The beneficial effects of this invention are as follows: By using a multi-camera array to collect surveillance videos from different perspectives, automatic target detection is performed on the dump trucks in the surveillance videos, and an information feature matrix of the dump trucks is constructed. Then, based on the information feature matrix, anomalies of the dump trucks in the surveillance videos are detected. This enables the acquisition of multi-angle videos from multiple cameras when the dump trucks are leaving the site, thereby obtaining videos of the dump trucks from different angles. This allows for subsequent judgment from different angles on issues such as whether the dump trucks have been cleaned and sealed. Based on this, an initial multi-angle information feature matrix of the dump trucks is constructed, and then compared with a preset standard information feature matrix of normal dump trucks of the same type to perform multi-angle target anomaly detection on the leaving dump trucks. This system uses detection to determine whether dump trucks are overloaded, uncleaned, or unsealed, ensuring accurate data and rapid identification of various violations. It improves both the accuracy and efficiency of dump truck monitoring, enabling automatic detection and identification of dump trucks and corresponding anomalies. This results in precise and efficient dump truck monitoring, reducing manual verification costs and false alarm rates. Ultimately, it enhances the digitalization of dump truck monitoring based on AI, contributing to smart city and digital governance. Attached Figure Description

[0010] Figure 1 A schematic diagram illustrating the overall concept and application environment of the AI-based surveillance video target detection method provided in this embodiment of the invention;

[0011] Figure 2 This is a flowchart illustrating the AI-based target detection method for surveillance videos provided in an embodiment of the present invention.

[0012] Figure 3 This is a schematic diagram of the first sub-process of the AI-based surveillance video target detection method provided in an embodiment of the present invention;

[0013] Figure 4 This is a schematic diagram of the second sub-process of the AI-based target detection method for surveillance videos provided in an embodiment of the present invention;

[0014] Figure 5 This is a schematic block diagram of an artificial intelligence-based surveillance video target detection system provided in an embodiment of the present invention. Detailed Implementation

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0016] This invention provides an artificial intelligence-based method and system for target detection in surveillance videos. The method and system can be applied to devices including but not limited to vehicle terminals, smartphones, tablets, servers, and cloud platforms, and can be used in scenarios including but not limited to monitoring and supervising the transportation of construction waste to detect targets in surveillance videos based on artificial intelligence.

[0017] The AI-based target detection method and system for surveillance videos provided in this invention can be applied to, for example... Figure 1 In the application environment. Please refer to Figure 1 , Figure 1 This diagram illustrates the overall concept and application environment of the AI-based target detection method for surveillance videos, as provided in this embodiment of the invention. Figure 1As shown, the first camera array, the second camera array, and the road monitoring points communicate with the backend server for monitoring the transportation of construction waste trucks. The server can receive exit monitoring videos from the first camera array, entry monitoring videos from the second camera array, the location information of the construction waste trucks, and video data from the road monitoring points. Based on the exit monitoring videos from the first camera array, it constructs an initial information feature matrix for the construction waste trucks to determine if the exiting trucks are normal. Based on the location information of the construction waste trucks and the video data from the road monitoring points, it monitors the real-time location of the construction waste trucks. When an abnormal location is detected, it combines the video data from the road monitoring points based on a time-series sequence to determine if the construction waste trucks are traveling along a preset transportation route and to identify any abnormalities such as dust or spillage. Based on the similarity between the final information feature matrix corresponding to the entry monitoring video and the initial information feature matrix of the exit monitoring video, it determines whether the construction waste trucks entering the disposal site are abnormal. The server can also send the detection results of abnormal construction waste trucks to supervisors and construction waste truck drivers via, but not limited to, onboard terminals and client apps. The system, based on a multi-camera array, collects surveillance videos from different angles to automatically detect dump trucks within the videos and constructs an information feature matrix for each truck. This allows for the detection of anomalies in the videos, and similarity detection is performed between the truck's exit feature matrix and its entry feature matrix at the disposal site to determine if a truck entering the site ultimately exhibits an anomaly. This system enables three-stage detection of the entire operation process—from exit to transportation to entry—solving the problem of low accuracy and efficiency in traditional methods that rely solely on snapshots for monitoring. Therefore, this AI-based intelligent dump truck monitoring system achieves precise and efficient monitoring of dump truck transportation, improving the digitalization of AI-based monitoring and thus enhancing smart city and digital governance. The client device can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. An APP (Application) typically refers to a software program running on a smartphone, tablet, or other mobile device. The server-side can be implemented using a cloud platform, a standalone server, or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0018] Example 1, please refer to Figure 1 and Figure 2 , Figure 1 A schematic diagram illustrating the overall concept and application environment of the AI-based surveillance video target detection method provided in this embodiment of the invention; Figure 2 This is a flowchart illustrating an artificial intelligence-based target detection method for surveillance videos, provided in an embodiment of the present invention. Figure 2As shown, in this embodiment, the method includes, but is not limited to, the following steps S21-S27:

[0019] S21. Based on a preset first camera array, collect exit monitoring videos of dump trucks leaving the site, the exit monitoring videos including the main exit monitoring video.

[0020] Explained, the pre-configured first camera array, also known as the preset first camera array, includes, but is not limited to, high-definition cameras, panoramic surveillance cameras, thermal imaging cameras, and dustproof and waterproof cameras. Appropriate cameras can be selected from existing camera types as needed. Furthermore, the preset first camera array can be deployed according to required layout. Figure 1 In the example, the preset first camera array includes a front-view camera 11, a right-side-view camera 12, a left-side-view camera 13, a top-view camera 14, and a rear-view camera 15. The front-view camera 11 captures monitoring video of the dump truck from the front, and the right-side-view camera 12 captures monitoring video of the dump truck from the right side, and so on. Furthermore, a camera array refers to a system composed of multiple cameras arranged in a specific layout, achieving functions that a single camera cannot accomplish through collaborative work. The term "first" in the first camera array is used only to distinguish different camera arrays and is not intended to limit the camera array. Other similar terms in this embodiment of the invention are similar.

[0021] Based on the above concept and deployment, and using a pre-set first camera array, exit monitoring videos are collected at the vehicle checkpoints corresponding to the construction sites from which the dump trucks leave the construction site. The exit monitoring videos include the main exit monitoring video, which refers to the monitoring video that can identify the type of dump truck among all exit monitoring videos. The main monitoring video can be, but is not limited to, a front-view monitoring video taken by a front-view camera and a rear-view monitoring video taken by a rear-view camera. The main monitoring video can be designated by relevant personnel based on experience, such as designating the aforementioned front-view monitoring video as the main monitoring video, or it can be automatically learned from unlabeled data based on unsupervised machine learning algorithms such as clustering.

[0022] S22. Based on the pre-trained dump truck detection model, detect whether the main monitoring video of the departure site contains a dump truck.

[0023] Explained, a pre-set dump truck detection model is prepared and pre-trained accordingly to obtain a pre-trained dump truck detection model. The pre-trained dump truck detection model refers to a pre-trained model that can be directly deployed in the application environment and is aimed at detecting dump trucks. The pre-trained dump truck detection model includes, but is not limited to, models based on YOLO, Faster R-CNN, and Transformer, and can also be selected from existing object detection models as needed.

[0024] Based on the above concept and setup, and using a pre-trained dump truck detection model, the system first detects whether the main monitoring video stream at the exit site contains dump trucks, i.e. whether there are dump trucks exiting at the vehicle checkpoint corresponding to the dump truck exit location.

[0025] S23. If the main monitoring video of the exit includes a dump truck, identify the vehicle identification and model type of the dump truck, and construct the initial information feature matrix of the dump truck based on the exit monitoring video and a preset feature matrix construction method.

[0026] Explain, the pre-set feature matrix construction method, i.e., the preset feature matrix construction method, refers to the way to construct a matrix from several features of the dump truck. The preset feature matrix construction method generally includes, but is not limited to, feature extraction of dump trucks (feature extraction from exit monitoring videos), feature selection (selecting the most distinctive and representative features), feature encoding (numerical encoding of category type features), feature standardization (unifying the scale of features with different dimensions), and matrix organization (organizing feature vectors into a matrix according to the feature order such as time series or event series).

[0027] Based on the above concept and setup, when the main monitoring video of the exit includes dump trucks, the system automatically identifies the vehicle identification of the dump trucks based on a preset vehicle identification recognition method. Vehicle identification includes, but is not limited to, license plates, which are generally license plates. Furthermore, based on a preset vehicle type recognition model, the system automatically identifies the vehicle type of the dump truck, including, but not limited to, U-shaped dump trucks, rectangular dump trucks, self-dumping trucks, 8x4 (rear eight-wheel drive), 6x2 (front four rear eight-wheel drive), and semi-trailer dump trucks. The preset vehicle identification recognition method refers to a pre-set method for identifying dump truck vehicle identification, including, but not limited to, image processing methods (such as license plate localization, character segmentation, and character recognition steps) and deep learning-based methods (such as YOLO-LPR-based models and DAN models (i.e., Deformable Attention Network)). The preset vehicle type recognition model refers to a pre-set model for identifying the vehicle type of the dump truck, including, but not limited to, object detection models corresponding to YOLO or PP-YOLOE models. It should be noted that, given the research and development capabilities and technical knowledge that those skilled in the art should possess, it is understandable that vehicle identification and vehicle type recognition can draw upon existing technologies, which will not be elaborated upon here. Furthermore, vehicle identification and vehicle type recognition can be independent modules or integrated into the aforementioned pre-trained dump truck detection model, which is not limited here.

[0028] Furthermore, based on the exit monitoring video and a preset feature matrix construction method, such as feature extraction, feature filtering, feature encoding, feature standardization, and matrix organization of the exit monitoring video, an initial information feature matrix corresponding to the dump truck is constructed. The initial information feature matrix represents the feature set of dump truck information when the dump truck leaves the dump site vehicle checkpoint. The term "initial" in the initial information feature matrix is ​​used to distinguish different information feature matrices, but is not used to limit different information feature matrices. Other similar terms in this embodiment of the invention are analogous. In addition, the elements in the initial information feature matrix represent, but are not limited to, information related to the dump truck and the transport of construction waste, including but not limited to the cleaning features and sealing features of the dump truck.

[0029] S24. Determine the preset standard information feature matrix corresponding to the vehicle type.

[0030] Explained as described above, vehicle types include, but are not limited to, U-shaped dump truck bodies, rectangular dump truck bodies, self-dumping dump truck bodies, eight-wheel (8×4 drive), four-wheel (6×2 drive), and semi-trailer dump trucks. For each type of dump truck, a corresponding standard information feature matrix is ​​pre-set, i.e., a preset standard information feature matrix. The preset standard information feature matrix corresponds to the aforementioned initial information feature matrix, meaning that the structure and elements of the preset standard information feature matrix are consistent and corresponding with those of the aforementioned initial information feature matrix. Thus, the initial information feature matrix and the preset standard information feature matrix are comparable. Furthermore, the preset standard information feature matrix represents a matrix composed of the corresponding features of the dump truck of this type under normal conditions when transporting waste soil. The elements in the preset standard information feature matrix represent, but are not limited to, the cleaning features and sealing features of the dump truck under normal conditions when transporting waste soil, corresponding to information related to the dump truck and the transported waste soil. Since each type of dump truck is different, the characteristics of different types of dump trucks when transporting construction waste are also different under normal circumstances. Characterization means converting abstract or complex entities (such as objects, concepts, and relationships) into symbolic forms that can be perceived, stored, or processed. Therefore, for each type of dump truck, a preset standard information feature matrix is ​​set in advance. Then, the initial information feature matrix of the dump truck of that type is compared with the corresponding preset standard information feature matrix to determine whether there is an anomaly in the current dump truck. This can further improve the accuracy of anomaly detection for dump trucks.

[0031] Based on the above concept and setup, when a dump truck is detected in the exit monitoring video and its type is determined, a preset standard information feature matrix corresponding to that type is determined, thereby establishing a standard for judging whether the dump truck detected in the exit monitoring video has any corresponding anomalies.

[0032] S25. Based on a preset matrix similarity judgment method, determine whether the initial information feature matrix is ​​similar to the preset standard information feature matrix;

[0033] S26. If the initial information feature matrix is ​​similar to the preset standard information feature matrix, it is determined that the target dump truck corresponding to the vehicle identifier is not abnormal.

[0034] S27. If the initial information feature matrix is ​​not similar to the preset standard multi-information feature matrix, the target dump truck is determined to be abnormal.

[0035] Explained, a pre-defined matrix similarity judgment method is used, i.e., a preset matrix similarity judgment method. The preset matrix similarity judgment method represents the way to judge the similarity between different matrices. The preset matrix similarity judgment method includes, but is not limited to, distance measurement methods (Euclidean distance, Manhattan distance, Mahalanobis distance, etc., the distance between matrices measures the difference, the larger the distance, the lower the similarity), similarity coefficient methods (cosine similarity, Pearson correlation coefficient, Jaccard similarity coefficient, etc., to calculate the similarity between matrices, the closer the value is to 1, the more similar they are), and deep learning-based matrix similarity classification (direct classification (end-to-end), similarity learning (Siamese Network), etc.).

[0036] Based on the above concept and setup, and using a preset matrix similarity judgment method, it is determined whether the initial information feature matrix is ​​similar to the preset standard information feature matrix. If the initial information feature matrix is ​​similar to the preset standard information feature matrix, it indicates that the cleaning and sealing features of the dump truck detected in the exit monitoring video, which are related to dump truck transportation, are similar to the corresponding features of this type of dump truck when transporting dump trucks under normal conditions. Therefore, it is determined that the target dump truck corresponding to the vehicle identification is normal, i.e., the cleaning and sealing features of the dump truck detected in the exit monitoring video are related to dump truck transportation. If no abnormalities are detected, the dump trucks detected by default are considered normal. Similarly, if the initial information feature matrix is ​​dissimilar to the preset standard multi-information feature matrix, the target dump truck is determined to be abnormal. That is, if the washing features, sealing features, and other features related to dump truck transportation are detected in the exit monitoring video, abnormalities are determined. The dump trucks detected by default are considered normal. Thus, based on the first camera array, monitoring videos from different angles are collected to automatically detect dump trucks in the monitoring videos and construct the information feature matrix of the dump trucks. Then, based on the information feature matrix, comprehensive target detection of abnormalities in the dump trucks in the monitoring videos is performed.

[0037] Further, based on a preset matrix similarity judgment method, determining whether the initial information feature matrix is ​​similar to the preset standard information feature matrix includes:

[0038] A deep learning-based pre-trained matrix similarity classification model is used to determine whether the initial information feature matrix is ​​similar to the preset standard information feature matrix.

[0039] Specifically, a deep learning-based matrix similarity classification model is pre-set and pre-trained to obtain a pre-trained deep learning-based matrix similarity classification model. This pre-trained model represents a pre-trained deep learning model that can be directly deployed in an application environment, targeting matrix similarity classification. Pre-trained matrix similarity classification models include, but are not limited to, models based on direct classification (end-to-end, such as Transformer models, convolutional neural networks, and MLP neural networks (Multilayer Perceptrons)), similarity learning (Siamese Network), and autoencoders. Direct classification (end-to-end) involves concatenating the initial matrix information feature matrix with a preset standard information feature matrix (e.g., channel overlay or horizontal concatenation), inputting it to a pre-trained matrix similarity classification model based on Transformer models, convolutional neural networks (CNNs), or MLP neural networks, and outputting the similarity classification result between the initial matrix information feature matrix and the preset standard information feature matrix (binary classification (normal / abnormal) or multi-class classification (abnormal type)). Similarity learning (Siamese Network) involves... The core idea of ​​a network is to train the network to extract features and then calculate the similarity. Specifically, for a Siamese neural network, the input is an initial information feature matrix and a preset standard information feature matrix, and the output is a similarity score between the two matrices. An autoencoder, on the other hand, refers to training an autoencoder using normal samples (preset standard information feature matrix).

[0040] When inputting the initial information feature matrix, the reconstruction error is calculated. If the error exceeds the threshold, it is judged as abnormal. Furthermore, when implementing the technical solution of this invention, the above-mentioned appropriate model can be selected as needed.

[0041] Based on the above concept and setup, the deep learning-based pre-trained matrix similarity classification model determines whether the initial information feature matrix is ​​similar to the preset standard information feature matrix. This fully utilizes the automation and adaptive characteristics of deep learning, automatically adapting to the diverse feature structures of the dump truck, such as cleaning and sealing features, contained in the initial information feature matrix. At the same time, through data-driven optimization, it automatically adapts to the complex monitoring video scenarios such as nonlinearity and high noise corresponding to the exit monitoring video, further improving the accuracy of target detection based on dump trucks in monitoring videos.

[0042] This invention, in its embodiments, automatically detects dump trucks in surveillance videos from different angles using a multi-camera array, constructs an information feature matrix for the dump trucks, and then uses this matrix to detect anomalies in the videos. This allows for multi-angle video capture from multiple cameras as dump trucks leave the site, enabling subsequent assessment of issues such as cleaning and sealing. An initial multi-angle information feature matrix is ​​then constructed and compared with a preset standard information feature matrix for normal dump trucks of the same type, thus enabling multi-angle anomaly detection for leaving dump trucks. This system helps determine whether dump trucks are overloaded, uncleaned, or unsealed, ensuring accurate data and rapid identification of various violations. It improves both the accuracy and efficiency of construction waste transportation supervision. The AI-based intelligent dump truck monitoring system enables automatic detection and identification of dump trucks and automatic detection of abnormal targets corresponding to violations, achieving precise and effective supervision of construction waste transportation. This creates an efficient and accurate supervision model, reduces manual verification costs, lowers the false judgment rate, and enhances the digitalization level of construction waste transportation supervision based on AI, thereby improving the level of smart cities and digital governance.

[0043] In one embodiment, please refer to Figure 1 and Figure 3 , Figure 3 This is a schematic diagram of the first sub-process of the AI-based target detection method for surveillance videos provided in an embodiment of the present invention. Figure 3 As shown, in this embodiment, after determining that the target dump truck corresponding to the vehicle identifier is normal, the process further includes:

[0044] S31. Monitor the real-time location of the target dump truck;

[0045] S32. If a location anomaly is detected, determine the location where the anomaly occurred and obtain the anomaly location;

[0046] S33. Determine the preset transportation route of the target dump truck;

[0047] S34. Based on the preset transportation route, select n consecutive road monitoring points covering the abnormal location;

[0048] S35. Extract the video data corresponding to the time period of the road monitoring point and generate a video sequence arranged in time order to obtain a road monitoring point video sequence based on time sequence, where n is a natural number;

[0049] S36. Based on the video sequence of the road monitoring points and a pre-trained dump truck detection model, detect whether the target dump truck appears in the video of each of the road monitoring points.

[0050] S37. If the above detection is positive, it is determined that the target dump truck is normal;

[0051] S38. If the above detection is negative, the target dump truck is determined to be abnormal.

[0052] For an explanatory purposes, please refer to [link / reference]. Figure 1 When the target dump truck begins transporting construction waste, its real-time location is monitored. Real-time location can be achieved using, but is not limited to, GPS (Global Positioning System), BeiDou positioning, base station positioning, or a combination of the above methods. Existing technologies can be used for real-time location monitoring, which will not be elaborated here. Generally, monitoring the real-time location of the target dump truck to detect behaviors such as speeding, illegal dumping, and route deviation is an efficient and accurate method.

[0053] Furthermore, the system monitors whether the real-time positioning of the target dump truck is abnormal. If no abnormality is detected, real-time positioning is used to monitor the target dump truck. If an abnormality is detected, the location of the abnormality is determined, and the abnormal location is obtained. The abnormal location indicates the location where the real-time positioning of the dump truck became abnormal; it is generally the last location where the real-time positioning of the target dump truck could be detected. Figure 1 The example shows the abnormal location.

[0054] Since the transportation routes of dump trucks are generally fixed routes set in advance, i.e., preset transportation routes, and road monitoring points are set in advance along these routes, these road monitoring points refer to video monitoring points set up on the roads along the preset transportation routes. Where possible, video data from public video monitoring systems such as traffic electronic police, security monitoring cameras, smart light pole integrated cameras, highway ETC gantries, and roadside monitoring can be used. Based on the preset transportation route of the target dump truck and the location and abnormal location of the road monitoring points, n consecutive road monitoring points covering the abnormal location are selected, i.e., n consecutive road monitoring points before and after the abnormal location are obtained. The abnormal location is between the n consecutive road monitoring points. Video data for the corresponding time period of the road monitoring points is extracted, i.e., video data when the target dump truck passes (has actually happened) or should pass (is predicted to pass) the road monitoring points and before and after them are extracted. A video sequence arranged in chronological order is generated, resulting in a road monitoring point video sequence based on the time sequence, where n is a natural number. Under normal circumstances, the road monitoring point video sequence represents the driving trajectory sequence of the target dump truck.

[0055] Based on the video sequences from road monitoring points and a pre-trained dump truck detection model, the system detects whether a target dump truck appears in the video of each monitoring point. This involves checking each frame of the video from each monitoring point until the target dump truck is detected or no frame is detected after checking all frames. The pre-trained dump truck detection model can be the same as described above. If the detection is positive, it indicates that the target dump truck passed through the corresponding road monitoring point within the specified time, and the truck is deemed normal. Conversely, if the detection is negative, it indicates that the target dump truck did not pass through the corresponding road monitoring point within the specified time. In particular, if the target dump truck is not detected in the video data of the road monitoring points after the abnormal location, it is determined that the target dump truck is abnormal. The target dump truck has risks such as signal interruption, deviation from the electronic fence, abnormal speed, and illegal dumping of construction waste. In other words, the target dump truck has abnormal risks. Therefore, after the real-time positioning of the construction waste fails, video analysis, detection and tracking are automatically triggered to realize the automatic switching between different detection modes of the dump truck. By dynamically switching the monitoring mode of the dump truck during transportation, a synergistic detection effect is achieved. Based on the time sequence of road monitoring point video sequences, video analysis, detection and tracking are performed to realize continuous spatiotemporal tracking of the target dump truck, which can improve the accuracy of target dump truck detection.

[0056] Furthermore, detecting whether the target dump truck appears in the video of each of the road monitoring points includes:

[0057] Determine the driving speed range of the target dump truck;

[0058] Based on the driving speed range and the preset transportation route, determine the target time range for the target dump truck to pass through each of the road monitoring points;

[0059] The system detects whether the target dump truck appears in the video of each road monitoring point within the corresponding target time range.

[0060] Specifically, since the target dump trucks have a maximum and minimum speed limit, the speed range of the target dump trucks is determined. Then, by determining the distance from the vehicle checkpoint corresponding to the dump truck's departure point to each road monitoring point, the target time range for the target dump truck's passage through each road monitoring point can be calculated. The target time range does not need to be too precise; it can even be extended beyond the calculated time range to obtain the target time range. This allows for a more reasonable estimation of the time range while better adapting to the complexity of driving conditions, enabling the detection of target dump trucks to adapt to complex driving environments and making the detection more accurate. Therefore, by limiting the target time range to monitor whether the target dump truck appears in the video of each road monitoring point, it is possible to avoid behaviors such as signal interruption, deviation from electronic fences, abnormal speed, and illegal dumping of dump trucks, further improving the accuracy and efficiency of dump truck transportation supervision.

[0061] Furthermore, the video data is video data from the rear external view of the target dump truck, and the pre-trained dump truck detection model includes a dump truck identification module and a sealing status analysis module; based on the pre-trained dump truck detection model, detecting whether the target dump truck appears in the video of each of the road monitoring points includes:

[0062] Based on the dump truck identification module, it is detected whether the target dump truck appears in the video of each of the road monitoring points;

[0063] When the target dump truck is detected to appear in the video of the road monitoring point, the sealing status analysis module detects whether the target dump truck has an abnormal seal. The abnormal seal includes at least one of the following: the sealing cover is not closed, the dump truck overflows, or the dump truck body is deformed.

[0064] Specifically, the video data mentioned above is video data from the external rear view of the target dump truck, that is, the video data mentioned above is road monitoring data collected from the external rear view of the target dump truck. Furthermore, the pre-trained dump truck detection model includes a dump truck recognition module and a sealing status analysis module. That is, the pre-trained dump truck detection model integrates these two modules and performs the following detections: The dump truck recognition module uses a deep learning-based target detection algorithm to determine whether a target dump truck appears in a single road monitoring point video. This module is based on models including, but not limited to, YOLO, Faster R-CNN, and Transformer models. The sealing status analysis module uses image semantic segmentation technology to detect whether the target dump truck has sealing anomalies, including but not limited to unclosed sealing covers, spilled construction waste, and deformed dump truck body structures. Image semantic segmentation classifies each pixel in an image into a predefined semantic category, thereby achieving a refined understanding and segmentation of the image. Image semantic segmentation includes, but is not limited to, fully convolutional networks, U-Net networks, Transformer-based models, and DeepLab semantic segmentation models, which can improve the accuracy of dump truck sealing detection.

[0065] Based on the above concept and setup, and using a pre-trained dump truck detection model, when detecting whether a target dump truck appears in the video of each road monitoring point, the following detection is performed: First, based on the dump truck identification module, it detects whether the target dump truck appears in the video of each road monitoring point; if the target dump truck is detected to be in the video of the road monitoring point, based on the sealing status analysis module, it detects whether the target dump truck has an abnormal seal. An abnormal seal includes at least one of the following: the sealing cover is not closed, the dump truck overflows, or the dump truck body is deformed. This allows for the monitoring of whether dump trucks deviate from the preset transportation route during transportation by combining dump truck positioning with the detection of road monitoring point videos based on time sequences. It can also detect whether the target dump truck is illegally dumping, causing dust pollution, or spilling dump truck waste. This approach not only balances the efficiency and accuracy of dump truck detection but also monitors whether dump trucks are illegally dumping, causing dust pollution, or spilling dump truck waste during transportation, thereby further improving the precision and effectiveness of dump truck transportation supervision.

[0066] This invention, in its embodiments, addresses the challenges of real-time positioning anomalies and failures, such as signal shielding or human interference, by combining real-time road monitoring video based on time-series data to determine whether the dump truck is traveling along a pre-set transportation route. This achieves a collaborative detection mechanism between the target dump truck's positioning and video time-series data. It leverages the efficiency and convenience of positioning to track and detect the dump truck's movement trajectory, and automatically triggers video analysis, detection, and tracking after positioning failure. This allows for the necessary automatic switching between different detection modes of the dump truck, achieving a synergistic and efficient detection effect through dynamic switching of the monitoring mode during transportation. This not only solves the two major pain points of existing technologies—"regulatory gaps after positioning failure" and "low efficiency of simple video analysis"—but also, by implementing a "positioning anomaly triggering mechanism + spatiotemporal video data fusion," fully utilizes the efficiency of positioning tracking and the accuracy of time-series video monitoring, thereby improving both the accuracy and efficiency of dump truck transportation supervision and enhancing the precision and effectiveness of dump truck transportation supervision.

[0067] In one embodiment, please refer to Figure 1 and Figure 4 , Figure 4 This is a schematic diagram of the second sub-process of the AI-based target detection method for surveillance videos provided in an embodiment of the present invention. (See diagram below.) Figure 4 As shown, in this embodiment, after determining that the target dump truck corresponding to the vehicle identifier is normal, the process further includes:

[0068] S41. Based on the preset second camera array, collect entry monitoring video of the dump truck entering the disposal site;

[0069] S42. Detect the target dump truck based on the entry monitoring video;

[0070] S43. When the target dump truck is detected, construct a final information feature matrix corresponding to the initial information feature matrix based on the entry monitoring video.

[0071] S44. Based on the pre-trained matrix similarity classification model, determine whether the final information feature matrix is ​​similar to the initial information feature matrix;

[0072] S45. If the final information feature matrix is ​​similar to the initial information feature matrix, it is determined that the target dump truck is normal.

[0073] S46. If the final information feature matrix is ​​not similar to the initial information feature matrix, the target dump truck is determined to be abnormal.

[0074] Explained, referring to the construction of the initial information feature matrix above, the difference between the initial and final information feature matrices is that the initial information feature matrix corresponds to the exit monitoring video of the dump truck at the vehicle checkpoint corresponding to the exit site, while the final information feature matrix corresponds to the entry monitoring video of the dump truck at the vehicle checkpoint corresponding to the entry site. Therefore, based on a preset second camera array, entry monitoring video of the dump truck entering the corresponding vehicle checkpoint of the disposal site is collected. When a target dump truck is detected, the final information feature matrix corresponding to the initial information feature matrix is ​​constructed based on the entry monitoring video. Here, "final" in the final information feature matrix corresponds to "initial" in the initial information feature matrix; they are antonyms. Then, based on... Based on a pre-trained matrix similarity classification model, the system determines whether the final information feature matrix is ​​similar to the initial information feature matrix. If the final information feature matrix is ​​similar to the initial information feature matrix, the target dump truck is deemed to be normal. If the final information feature matrix is ​​not similar to the initial information feature matrix, the target dump truck is deemed to be abnormal. This enables the final detection of dump trucks entering the disposal site. By performing similarity detection based on the information feature matrix of the truck leaving the disposal site and the information feature matrix of the truck entering the disposal site, the system determines whether the dump truck is ultimately abnormal upon entering the disposal site. This achieves detection at both the "leaving" and "entering" ends of the dump truck operation and enables full online detection of the entire operation process from the construction site source to the disposal site, corresponding to the "leaving-transporting-entering" of the dump truck.

[0075] This invention, in its embodiments, determines whether a target dump truck is abnormal by checking whether its final information feature matrix matches its initial information feature matrix when the target dump truck enters the disposal site. If the final and initial information feature matrices match, the target dump truck is considered normal; otherwise, it is considered abnormal. This method compares multi-dimensional information of the dump truck at both the departure and arrival points to check for abnormalities. Furthermore, it enables three-stage detection of the entire operation process of dump trucks—"departure-transportation-arrival"—ensuring both the accuracy of information and the safety of the dump truck. The system should provide accurate and error-free data, be able to quickly identify different violations, and improve both the accuracy and efficiency of construction waste transportation supervision. It should also enable seamless and compact supervision of the entire operation process of construction waste trucks, from departure to arrival. Through an AI-based intelligent construction waste truck monitoring system, precise and effective supervision of construction waste transportation can be achieved, creating an efficient and accurate supervision model. This reduces manual verification costs and lowers the error rate, improving the digitalization level of construction waste transportation supervision based on AI, thereby enhancing the level of smart city and digital governance.

[0076] In one embodiment, the exit monitoring video includes at least two of the following perspectives: a frontal view monitoring video, a side view monitoring video, a rear view monitoring video, and a top view monitoring video; based on the exit monitoring video and a preset feature matrix construction method, an initial information feature matrix of the dump truck is constructed, including:

[0077] Based on the viewpoint videos included in the exit monitoring video, the following information features are extracted from the corresponding viewpoint videos:

[0078] A pre-trained video feature extraction model based on deep learning is used to extract information features from the front-view surveillance video to obtain initial front-view information features.

[0079] Based on the pre-trained video feature extraction model, information features of the side-view surveillance video are extracted to obtain initial side-view information features.

[0080] Based on the pre-trained video feature extraction model, information features of the rear view surveillance video are extracted to obtain initial rear view information features;

[0081] Based on the pre-trained video feature extraction model, information features of the top-view surveillance video are extracted to obtain initial top-view information features;

[0082] At least two of the initial frontal view information features, initial side view information features, initial rear view information features, and initial top view information features extracted above are combined into a matrix according to a preset view order to obtain the initial information feature matrix corresponding to the dump truck.

[0083] Explain, a video feature extraction model is pre-set and pre-trained to obtain a pre-trained video feature extraction model. The pre-trained video feature extraction model represents a pre-trained model that can be directly deployed in the application environment and aims at feature extraction from the aforementioned surveillance video. The pre-trained video feature extraction model includes, but is not limited to, models based on YOLO, Faster R-CNN, and Transformer. It can also be selected from existing object detection models as needed. The features of the surveillance video include, but are not limited to, cleaning features, sealing features, spillage and leakage features, and overloading features (such as large-volume construction waste protruding from the truck bed).

[0084] Based on the above concept and setup, the exit monitoring video includes at least two of the following perspectives: front view monitoring video of the dump truck, side view monitoring video, rear view monitoring video, and top view monitoring video, as described above.

[0085] Then, when constructing the initial information feature matrix corresponding to the dump truck, the corresponding exit monitoring video includes the front view monitoring video, so the information feature extraction of the corresponding front view monitoring video is performed. The exit monitoring video includes both front view monitoring video and rear view monitoring video, so the information feature extraction of the corresponding front view monitoring video and the information feature extraction of the rear view monitoring video are performed respectively, and so on. Thus, at least two of the following information feature extractions of the view videos are performed: using a pre-trained video feature extraction model based on deep learning to extract the information features of the front view monitoring video to obtain the initial front view information features. The initial front view information features represent the dump truck feature set from the front view of the dump truck when it leaves the dump construction site vehicle checkpoint. The word "initial" in the initial front view information features is used to distinguish different front view information features, but is not used to limit different front view information features. Other similar terms in the embodiments of this invention are similar. Similarly, based on the pre-trained video feature extraction model, information features of the side view monitoring video are extracted to obtain the initial side view information features; information features of the rear view monitoring video are extracted to obtain the initial rear view information features; and information features of the top view monitoring video are extracted to obtain the initial top view information features, where the top view is the roof view.

[0086] Finally, at least two of the initial frontal view information features, initial side view information features, initial rear view information features, and initial top view information features extracted above are combined into a matrix according to a preset view order to obtain the initial information feature matrix corresponding to the dump truck. The preset view order can be set in advance.

[0087] This invention, through the acquisition of videos from different perspectives of dump trucks and the extraction of corresponding information features, constructs an initial information feature matrix corresponding to the dump trucks. This matrix is ​​then used to uniformly determine whether the dump trucks are clean and sealed from different angles, thereby enabling multi-angle target anomaly detection of the departing dump trucks. This allows for the identification of issues such as overloading, lack of cleaning, and lack of sealing, ensuring accurate data and rapid identification of various violations. This improves both the accuracy and efficiency of dump truck transportation supervision. By implementing an AI-based intelligent dump truck monitoring system, the system achieves automatic detection and identification of dump trucks and automatic anomaly detection corresponding to violations, thus realizing precise and effective dump truck transportation supervision and creating a highly efficient and accurate dump truck transportation supervision model.

[0088] In one embodiment, at least two of the extracted initial frontal view information features, initial side view information features, initial rearal view information features, and initial top view information features are arranged into a matrix according to a preset view order to obtain the initial information feature matrix corresponding to the dump truck, including:

[0089] Determine the actual load capacity of the dump truck;

[0090] At least two of the extracted initial frontal view information features, initial side view information features, initial rear view information features, and initial top view information features, along with the actual load, are combined into a matrix according to a preset view order and a preset view load order to obtain the initial information feature matrix corresponding to the dump truck.

[0091] Explanatoryly, the actual load of the dump truck is determined. The actual load of the dump truck can be obtained by means of, but not limited to, direct weighing technology (installing a weighing sensor on the vehicle chassis to measure the load in real time), pressure sensor (monitoring the pressure change of the hydraulic suspension system to estimate the load), and strain gauge technology (installing strain gauges on key parts of the frame to measure the deformation and calculate the load). Existing technical means can be used as a reference, and will not be elaborated here.

[0092] At least two of the extracted initial front view information features, initial side view information features, initial rear view information features, and initial top view information features are combined with the actual load and arranged into a matrix according to a preset view order and a preset view load order. That is, the view information features and the load features corresponding to the actual load are arranged into a matrix in a certain order to obtain the initial information feature matrix corresponding to the dump truck. Thus, the actual load of the dump truck is used as the element and composition of the initial information feature matrix, so as to realize the comprehensive detection of abnormal targets of the dump truck by combining the actual load of the dump truck with video features, which can improve the efficiency of abnormal target detection of dump trucks.

[0093] This invention, through its embodiments, combines video features and actual load characteristics of dump trucks to form an information feature matrix. This matrix is ​​then used to detect abnormal targets in the monitored video, enabling comprehensive detection of issues such as overloading, lack of cleaning, and unsealing of dump trucks. This AI-based intelligent dump truck monitoring system achieves automatic detection and identification of vehicle targets and abnormal targets in the monitored video, thereby enhancing the precision and efficiency of dump truck transportation supervision. This creates an efficient and accurate supervision model for dump truck transportation, improving the digitalization level of AI-based dump truck transportation supervision and ultimately enhancing the level of smart cities and digital governance.

[0094] In one embodiment, the preset standard information feature matrix includes a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix, and the elements in the preset standard information feature upper limit matrix, the preset standard information feature lower limit matrix, and the initial information feature matrix are all quantized features; based on a preset matrix similarity judgment method, determining whether the initial information feature matrix and the preset standard information feature matrix are similar includes:

[0095] The initial information feature matrix is ​​compared element-by-element with the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix.

[0096] Determine whether each element in the initial information feature matrix falls within the upper and lower limit interval formed by the corresponding upper and lower limits;

[0097] If the above judgment is not true, it is determined that the initial information feature matrix is ​​not similar to the preset standard information feature matrix;

[0098] If the above judgment is true, the initial information feature matrix is ​​determined to be similar to the preset standard information feature matrix.

[0099] Interpretatively, the abnormal targets that can be detected by the monitoring videos of dump trucks include, but are not limited to, exceeding limits (dump height exceeding the truck bed panels or obstructing the view), inadequate sealing (unsealed transport, such as tarpaulin not covering, excessive gaps in the truck bed), vehicle deformation (tilted or dented truck bed), obscured or damaged license plates, and lack of cleaning (mud on the road). The characteristics corresponding to these anomalies can all be quantified, and their upper and lower limits can be determined. Quantified features represent features that can be expressed numerically (discrete or continuous). They are typically standardized data extracted from raw data through measurement, calculation, or statistical methods. They are used to objectively describe the attributes of objects and support mathematical operations, comparisons, or modeling. Examples of quantified features and their upper and lower limits are shown in Table 1 below.

[0100] Table 1

[0101] Abnormal behavior of dump trucks Quantitative features Exceeding limits Stacking height exceeds proportion The tarpaulin was not covered. Tarpaulin coverage ratio (0-1) The gaps between the carriages are too large Gap width (cm) The carriage tilted and dented. Symmetry score (0-1) License plate obscured or damaged License plate recognition confidence level (0-1) Unwashed vehicle body mud coverage

[0102] Therefore, by quantifying the characteristics based on the abnormal behavior of the dump truck, the upper and lower limits of the quantified characteristics can be determined. The upper and lower limits can be used to determine whether the dump truck is abnormal. Furthermore, for cases where only an upper or lower limit exists, the corresponding lower or upper limit can be filled with 0. For example, the lower limit of the quantified characteristic "stacking height" that exceeds the limit can be set to "0".

[0103] Among them, 1) Excess pile height ratio is defined as "(actual pile height - truck bed baffle height) / truck bed baffle height". The detection method includes, but is not limited to, the following: based on image segmentation, identify the baffle edge and the outline of the slag, and calculate the excess ratio; 2) Tarpaulin coverage ratio is defined as the ratio of the area of ​​the top of the slag truck bed effectively covered by the tarpaulin to the total area of ​​the top of the slag truck bed, usually expressed as a percentage (0%~100%). It can be detected based on image segmentation, that is, by taking a top view of the top of the slag truck bed through the vehicle-mounted camera, and using image semantic segmentation, such as using a deep learning model (such as U-Net, DeepLab) to segment the tarpaulin area and the slag truck bed area in the image, outputting a mask image of tarpaulin pixels (green) and uncovered pixels (red), and then counting the proportion of tarpaulin pixels to the total number of pixels on the top of the slag truck bed; 3) Gap width is defined as the maximum physical gap distance between the tarpaulin of the slag truck bed and the baffle and edge of the slag truck bed, usually expressed in centimeters (cm) or pixels (needs calibration conversion). It can be detected based on deep learning models. 4) End-to-end regression for detection, for example, selecting a model "lightweight CNN (such as MobileNet) + regression head" and training the model "inputting a local image of the carriage and outputting the gap width (unit: cm)", then deploying it to the application environment, inputting the collected local image of the current carriage, performing real-time prediction, and directly outputting the gap width value; 5) Symmetry score (0~1), defined as the calculation of mirror symmetry through key points (such as corners and edges) on the left and right sides of the carriage, where 1 indicates complete symmetry. For key point detection, geometric features of the carriage can be extracted using YOLO or OpenCV; 6) License plate recognition confidence, defined as the confidence of the OCR model in recognizing license plate characters (such as EasyOCR output probability). The detection method is to use the OCR model to recognize the license plate in real time and output the confidence; 7) Body mud coverage rate, defined as the percentage of pixels on the body surface covered by mud. The detection method can be to identify the mud area on the body based on an image semantic segmentation model (such as U-Net).

[0104] It should be noted that the quantitative characteristics and their detection methods corresponding to the above-mentioned pile height excess ratio, tarpaulin coverage ratio, gap width, symmetry score, tilt angle, local indentation depth, license plate recognition confidence, and mud coverage of the vehicle body are merely illustrative descriptions. For those skilled in the art, given their expected research and development capabilities and technical understanding, it is understandable that other quantitative characteristics or detection methods can be used for the illustrative quantitative characteristics and their respective detection methods of abnormal performance of dump trucks, including but not limited to exceeding limits, poor sealing, vehicle body deformation, license plate obstruction or damage, and lack of cleaning. No limitation is made here.

[0105] As described above, the preset standard information feature matrix includes a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix. The preset standard information feature upper limit matrix represents the upper limit of the corresponding quantitative features of a dump truck of a certain model type under normal conditions when transporting construction waste. The elements of the preset standard information feature upper limit matrix represent, but are not limited to, the upper limit of the quantitative features related to the dump truck and the transported construction waste, after quantifying the cleaning and sealing features of the dump truck under normal conditions when transporting construction waste. Similarly, the preset standard information feature lower limit matrix represents the lower limit of the corresponding quantitative features of a dump truck of a certain model type under normal conditions when transporting construction waste. The elements of the preset standard information feature lower limit matrix represent, but are not limited to, the lower limit of the quantitative features related to the dump truck and the transported construction waste, after quantifying the cleaning and sealing features of the dump truck under normal conditions when transporting construction waste. Furthermore, all elements in the preset standard information feature upper limit matrix, the preset standard information feature lower limit matrix, and the initial information feature matrix are quantitative features.

[0106] Based on the above concept and setup, and using a preset matrix similarity judgment method, when determining whether the initial information feature matrix is ​​similar to the preset standard information feature matrix, the following steps are performed:

[0107] The initial information feature matrix is ​​compared element by element with the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix; it is determined whether each element in the initial information feature matrix falls within the upper and lower limit intervals formed by the corresponding upper and lower limits; if the above determination is negative, the initial information feature matrix is ​​determined to be dissimilar to the preset standard information feature matrix; if the above determination is positive, the initial information feature matrix is ​​determined to be similar to the preset standard information feature matrix.

[0108] This invention, through setting a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix, compares the elements in the initial information feature matrix with the elements in the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix respectively, to determine whether each element in the initial information feature matrix falls within the upper and lower limit intervals formed by the corresponding upper and lower limits. This controls the elements in the initial information feature matrix within the upper and lower limit intervals formed by the corresponding elements in the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix. By quantifying the corresponding features, the flexibility and accuracy of comprehensive detection of construction waste trucks can be achieved. Thus, through an intelligent construction waste truck monitoring system based on artificial intelligence, automatic comprehensive detection and identification of vehicle targets and abnormal targets in monitoring videos of construction waste trucks can be realized, thereby achieving precise and efficient construction waste transportation supervision, creating an efficient and accurate construction waste transportation supervision model, and improving the digital level of construction waste transportation supervision based on artificial intelligence, thereby improving the level of smart cities and digital governance.

[0109] In one embodiment, after determining that the target dump truck corresponding to the vehicle identifier is abnormal, the method further includes:

[0110] The abnormal information of the target dump truck is sent to a preset terminal so that relevant personnel can be aware of the abnormal status of the target dump truck in a timely manner. The preset terminal includes at least one of the following: vehicle terminal, APP. The relevant personnel include at least one of the following: supervisors, dump truck drivers, and dump truck transportation responsible persons.

[0111] Explanatoryly, after determining that the target dump truck corresponding to the vehicle identification is abnormal, the abnormality information of the target dump truck is sent to a preset terminal. The preset terminal includes at least one of the following: vehicle-mounted terminal and client APP. The relevant personnel include at least one of the following: supervisory personnel, dump truck driver, and dump truck transport responsible person, so that the relevant personnel can be aware of the abnormal status of the target dump truck in a timely manner and can avoid the adverse consequences caused by violations such as overloading, dust pollution, dumping, and illegal dumping.

[0112] This invention, through its embodiments, promptly sends information about abnormalities in target dump trucks to a preset terminal, enabling relevant personnel to be aware of the trucks' abnormal status in a timely manner. This not only provides timely notification of dump truck anomalies but also serves as a warning against violations such as overloading, dust pollution, dumping, and unauthorized dumping, thus preventing such incidents from occurring in the first place. This achieves the regulatory objective of preventing violations in dump truck transportation, thereby improving both the accuracy and efficiency of dump truck transportation supervision. Furthermore, this AI-based intelligent dump truck monitoring system achieves precise and effective supervision of dump truck transportation, enhancing the digitalization level of AI-based dump truck transportation supervision and ultimately improving the level of smart city and digital governance.

[0113] It should be noted that the artificial intelligence-based target detection methods for surveillance videos described in the above embodiments can be recombined as needed to obtain combined implementation schemes, but all are within the protection scope claimed by this invention.

[0114] In one embodiment, an AI-based surveillance video target detection system is provided, which corresponds one-to-one with the AI-based surveillance video target detection method described in the above embodiments. Please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic block diagram of an artificial intelligence-based target detection system for surveillance videos, provided in an embodiment of the present invention. Figure 5As shown, the AI-based surveillance video target detection system 50 includes a first acquisition module 51, a first detection module 52, a first recognition module 53, a first determination module 54, a first judgment module 55, and a first determination module 56. The detailed descriptions of each functional module are as follows: The first acquisition module 51 is used to acquire exit monitoring videos of dump trucks based on a preset first camera array, the exit monitoring videos including the main exit monitoring video; the first detection module 52 is used to detect whether the main exit monitoring video contains a dump truck based on a pre-trained dump truck detection model; the first recognition module 53 is used to identify the vehicle identification and model type of the dump truck if the above detection is positive, and to construct an initial information feature matrix of the dump truck based on the exit monitoring video and a preset feature matrix construction method; the first determination module 54 is used to determine the preset standard information feature matrix corresponding to the model type; the first judgment module 55 is used to determine whether the initial information feature matrix is ​​similar to the preset standard information feature matrix based on a preset matrix similarity judgment method; the first determination module 56 is used to determine that the target dump truck corresponding to the vehicle identification is normal if the above judgment is positive.

[0115] In one embodiment, the surveillance video target detection system 50 further includes: a first monitoring module for monitoring the real-time positioning of the target dump truck; a second determination module for determining the location of the anomaly if an anomaly is detected; a third determination module for determining the preset transportation route of the target dump truck; a first filtering module for filtering out n consecutive road monitoring points covering the anomaly location based on the preset transportation route; a first generation module for extracting video data of the corresponding time period of the road monitoring points and generating a video sequence arranged in chronological order to obtain a road monitoring point video sequence based on the time sequence, where n is a natural number; a second detection module for detecting whether the target dump truck appears in the video of each road monitoring point based on the road monitoring point video sequence and a pre-trained dump truck detection model; and a second judgment module for determining that the target dump truck has no anomaly if the above detection is positive.

[0116] In one embodiment, the second detection module includes: a first determining submodule, configured to determine the driving speed range of the target dump truck; a second determining submodule, configured to determine the target time range for the target dump truck to pass through each of the road monitoring points based on the driving speed range and the preset transportation route; and a first detection submodule, configured to detect whether the target dump truck appears in the video of each of the road monitoring points within the corresponding target time range.

[0117] In one embodiment, the video data is video data from the rear external view of the target dump truck, and the pre-trained dump truck detection model includes a dump truck identification module and a sealing status analysis module; the second detection module includes: a second detection submodule, used to detect whether the target dump truck appears in the video of each of the road monitoring points based on the dump truck identification module; and a third detection submodule, used to detect whether the target dump truck has an abnormal seal based on the sealing status analysis module when the target dump truck is detected to appear in the video of the road monitoring point, wherein the abnormal seal includes at least one of the following: the sealing cover is not closed, the dump truck overflows, or the dump truck body structure is deformed.

[0118] In one embodiment, the surveillance video target detection system 50 further includes: a second acquisition module, used to acquire entry surveillance video of a dump truck entering a disposal site based on a preset second camera array; a third detection module, used to detect the target dump truck based on the entry surveillance video; a first construction module, used to construct a final information feature matrix corresponding to the initial information feature matrix based on the entry surveillance video when the target dump truck is detected; a second judgment module, used to determine whether the final information feature matrix is ​​similar to the initial information feature matrix based on the pre-trained matrix similarity classification model; and a third determination module, used to determine that the target dump truck is normal if the final information feature matrix is ​​similar to the initial information feature matrix.

[0119] In one embodiment, the exit monitoring video includes at least two of the following perspectives: a monitoring video from the front of the dump truck, a monitoring video from the side, a monitoring video from the rear, and a monitoring video from the top; the first identification module 53 includes:

[0120] The first extraction submodule is used to extract information features of the corresponding viewpoint videos based on the viewpoint videos included in the exit monitoring video: extracting information features of the front view monitoring video based on a deep learning-based pre-trained video feature extraction model to obtain initial front view information features; extracting information features of the side view monitoring video based on the pre-trained video feature extraction model to obtain initial side view information features; extracting information features of the rear view monitoring video based on the pre-trained video feature extraction model to obtain initial rear view information features; and extracting information features of the top view monitoring video based on the pre-trained video feature extraction model to obtain initial top view information features.

[0121] The first sub-module is used to form a matrix by combining at least two of the extracted initial frontal view information features, initial side view information features, initial rear view information features, and initial top view information features into a matrix according to a preset view order, so as to obtain the initial information feature matrix corresponding to the dump truck.

[0122] In one embodiment, the first component submodule includes: a second determining submodule, used to determine the actual load of the dump truck; and a second component submodule, used to combine at least two of the extracted initial front view information features, initial side view information features, initial rear view information features, and initial top view information features with the actual load, and form a matrix according to a preset view order and a preset view load order to obtain the initial information feature matrix corresponding to the dump truck.

[0123] In one embodiment, the preset standard information feature matrix includes a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix, and the elements in the preset standard information feature upper limit matrix, the preset standard information feature lower limit matrix, and the initial information feature matrix are all quantized features; the first judgment module 55 includes: a first comparison submodule, used to compare the initial information feature matrix with the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix element by element; a first judgment submodule, used to determine whether each element in the initial information feature matrix falls within the upper and lower limit intervals formed by the corresponding upper and lower limits; and a first determination submodule, used to determine that the initial information feature matrix and the preset standard information feature matrix are not similar if the above determination is negative.

[0124] In one embodiment, the first judgment module 55 is specifically used to determine whether the initial information feature matrix is ​​similar to the preset standard information feature matrix using a deep learning-based pre-trained matrix similarity classification model.

[0125] This invention provides an artificial intelligence-based surveillance video target detection system. By using a multi-camera array to collect surveillance videos from different perspectives, it automatically detects dump trucks in the videos and constructs an information feature matrix for each dump truck. Based on this matrix, it then detects anomalies in the dump trucks within the surveillance videos. This improves both the accuracy and efficiency of dump truck transportation supervision. Through this AI-based intelligent dump truck monitoring system, it achieves automatic detection and identification of dump trucks and automatic anomaly detection corresponding to violations, thereby realizing precise and effective supervision of dump truck transportation.

[0126] Specific limitations regarding AI-based surveillance video target detection systems can be found in the above section on AI-based surveillance video target detection methods, and will not be repeated here. The modules in the aforementioned AI-based surveillance video target detection system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, allowing the processor to call and execute the corresponding operations of each module.

[0127] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] The software tools or components not belonging to our company that appear in the embodiments of this invention are merely illustrative examples and do not represent actual use.

[0129] The data collection in this embodiment of the invention complies with the requirements of relevant laws and regulations, such as China's Personal Information Protection Law, GDPR (General Data Protection Regulation of the European Union), or information security standards of other countries and regions.

[0130] The data collection in this embodiment of the invention complies with the requirements of relevant laws and regulations, such as GDPR (General Data Protection Regulation of the European Union) or other national and regional information security standards.

[0131] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A target detection method for surveillance videos based on artificial intelligence, characterized in that, include: Based on a preset first camera array, exit monitoring videos of dump trucks are collected, including the main exit monitoring video. Based on a pre-trained dump truck detection model, detect whether the main monitoring video of the departure site contains a dump truck; If the above detection is positive, identify the vehicle identification and model type of the dump truck, and construct an initial information feature matrix of the dump truck based on the exit monitoring video and a preset feature matrix construction method. The initial information feature matrix includes cleaning features, sealing features, spillage and leakage features, and overload features. Determine the preset standard information feature matrix corresponding to the vehicle type. The preset standard information feature matrix represents a matrix composed of the corresponding features of the dump truck of the corresponding vehicle type under normal conditions when transporting dump truck waste. The structure and elements of the preset standard information feature matrix are consistent with those of the initial information feature matrix. Based on a preset matrix similarity judgment method, it is determined whether the initial information feature matrix is ​​similar to the preset standard information feature matrix; If the above judgment is true, it is determined that the target dump truck corresponding to the vehicle identification is normal; The exit monitoring video includes at least two of the following perspectives: front view monitoring video, side view monitoring video, rear view monitoring video, and top view monitoring video; based on the exit monitoring video and a preset feature matrix construction method, an initial information feature matrix of the dump truck is constructed, including: Based on the viewpoint videos included in the exit monitoring video, the following information features are extracted from the corresponding viewpoint videos: A pre-trained video feature extraction model based on deep learning is used to extract information features from the front-view surveillance video to obtain initial front-view information features. Based on the pre-trained video feature extraction model, information features of the side-view surveillance video are extracted to obtain initial side-view information features. Based on the pre-trained video feature extraction model, information features of the rear view surveillance video are extracted to obtain initial rear view information features; Based on the pre-trained video feature extraction model, information features of the top-view surveillance video are extracted to obtain initial top-view information features; At least two of the initial frontal view information features, initial side view information features, initial rear view information features, and initial top view information features extracted above are combined into a matrix according to a preset view order to obtain the initial information feature matrix corresponding to the dump truck.

2. The AI-based target detection method for surveillance videos as described in claim 1, characterized in that, After determining that the target dump truck corresponding to the vehicle identifier is normal, the process also includes: Monitor the real-time location of the target dump truck; If a location anomaly is detected, determine the location where the anomaly occurred and obtain the anomaly location; Determine the preset transportation route for the target dump truck; Based on the preset transportation route, n consecutive road monitoring points covering the abnormal location are selected; Extract video data corresponding to the time period of the road monitoring points and generate a video sequence arranged in chronological order to obtain a road monitoring point video sequence based on time sequence, where n is a natural number; Based on the video sequence of the road monitoring points and a pre-trained dump truck detection model, detect whether the target dump truck appears in the video of each of the road monitoring points; If the above detection is positive, the target dump truck is determined to be normal.

3. The AI-based target detection method for surveillance videos as described in claim 2, characterized in that, Detecting whether the target dump truck appears in the video of each of the road monitoring points includes: Determine the driving speed range of the target dump truck; Based on the driving speed range and the preset transportation route, determine the target time range for the target dump truck to pass through each of the road monitoring points; The system detects whether the target dump truck appears in the video of each road monitoring point within the corresponding target time range.

4. The AI-based target detection method for surveillance videos as described in claim 2, characterized in that, The video data is video data from the rear external view of the target dump truck, and the pre-trained dump truck detection model includes a dump truck recognition module and a sealing status analysis module. Based on a pre-trained dump truck detection model, the system detects whether the target dump truck appears in the video of each of the road monitoring points, including: Based on the dump truck identification module, it is detected whether the target dump truck appears in the video of each of the road monitoring points; When the target dump truck is detected to appear in the video of the road monitoring point, the sealing status analysis module detects whether the target dump truck has an abnormal seal. The abnormal seal includes at least one of the following: the sealing cover is not closed, the dump truck overflows, or the dump truck body is deformed.

5. The AI-based target detection method for surveillance videos as described in claim 1, characterized in that, After determining that the target dump truck corresponding to the vehicle identifier is normal, the process also includes: Based on a pre-set second camera array, entry monitoring videos of dump trucks entering the disposal site are collected; The target dump truck is detected based on the entry monitoring video; When the target dump truck is detected, a final information feature matrix corresponding to the initial information feature matrix is ​​constructed based on the entry monitoring video. Based on a pre-trained matrix similarity classification model, it is determined whether the final information feature matrix is ​​similar to the initial information feature matrix; If the final information feature matrix is ​​similar to the initial information feature matrix, it is determined that the target dump truck is normal.

6. The AI-based target detection method for surveillance videos as described in claim 1, characterized in that, At least two of the extracted initial frontal view information features, initial side view information features, initial rear view information features, and initial top view information features are arranged into a matrix according to a preset view order to obtain the initial information feature matrix corresponding to the dump truck, including: Determine the actual load capacity of the dump truck; At least two of the extracted initial frontal view information features, initial side view information features, initial rear view information features, and initial top view information features, along with the actual load, are combined into a matrix according to a preset view order and a preset view load order to obtain the initial information feature matrix corresponding to the dump truck.

7. The AI-based target detection method for surveillance videos as described in claim 1, characterized in that, The preset standard information feature matrix includes a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix, and the elements in the preset standard information feature upper limit matrix, the preset standard information feature lower limit matrix and the initial information feature matrix are all quantized features; Based on a preset matrix similarity judgment method, the method determines whether the initial information feature matrix is ​​similar to the preset standard information feature matrix, including: The initial information feature matrix is ​​compared element-by-element with the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix. Determine whether each element in the initial information feature matrix falls within the upper and lower limit interval formed by the corresponding upper and lower limits; If the above judgment is not true, it is determined that the initial information feature matrix is ​​not similar to the preset standard information feature matrix.

8. The AI-based target detection method for surveillance videos as described in claim 1, characterized in that, Based on a preset matrix similarity judgment method, the method determines whether the initial information feature matrix is ​​similar to the preset standard information feature matrix, including: A deep learning-based pre-trained matrix similarity classification model is used to determine whether the initial information feature matrix is ​​similar to the preset standard information feature matrix.

9. A target detection system for surveillance video based on artificial intelligence, characterized in that, include: The first acquisition module is used to acquire exit monitoring videos of dump trucks based on a preset first camera array, the exit monitoring videos including the main exit monitoring video. The first detection module is used to detect whether the main monitoring video of the departure site contains a dump truck based on a pre-trained dump truck detection model. The first identification module is used to identify the vehicle identification and model type of the dump truck if the above detection is true, and to construct an initial information feature matrix of the dump truck based on the exit monitoring video and a preset feature matrix construction method. The initial information feature matrix includes cleaning features, sealing features, spillage and leakage features, and overload features. The first determining module is used to determine the preset standard information feature matrix corresponding to the vehicle type. The preset standard information feature matrix represents a matrix composed of the corresponding features of the dump truck of the corresponding vehicle type under normal conditions when transporting dump truck waste. The preset standard information feature matrix is ​​consistent with the structure and elements of the initial information feature matrix. The first judgment module is used to determine whether the initial information feature matrix is ​​similar to the preset standard information feature matrix based on a preset matrix similarity judgment method. The first determination module is used to determine that the target dump truck corresponding to the vehicle identifier is normal if the above determination is true. The exit monitoring video includes at least two of the following perspectives: front view monitoring video of the dump truck, side view monitoring video, rear view monitoring video, and top view monitoring video; the first recognition module includes: The first extraction submodule is used to extract information features of the corresponding viewpoint videos based on the viewpoint videos included in the exit monitoring video: extracting information features of the front view monitoring video based on a deep learning-based pre-trained video feature extraction model to obtain initial front view information features; extracting information features of the side view monitoring video based on the pre-trained video feature extraction model to obtain initial side view information features; extracting information features of the rear view monitoring video based on the pre-trained video feature extraction model to obtain initial rear view information features; and extracting information features of the top view monitoring video based on the pre-trained video feature extraction model to obtain initial top view information features. The first sub-module is used to form a matrix by combining at least two of the extracted initial frontal view information features, initial side view information features, initial rear view information features, and initial top view information features into a matrix according to a preset view order, so as to obtain the initial information feature matrix corresponding to the dump truck.

Images