Monitoring video target detection method and system based on artificial intelligence

Abstract

The invention belongs to the technical field of artificial intelligence, provides a monitoring video target detection method and system based on artificial intelligence, and aims to solve the problem of low accuracy and high efficiency of muck transportation supervision technical means in the traditional technology. According to the method, the monitoring video of the muck truck is acquired, automatic target detection is performed on the muck truck in the monitoring video, the information feature matrix of the muck truck is constructed, and target detection is performed on the abnormity of the muck truck in the monitoring video according to the information feature matrix, so that the monitoring accuracy of muck transportation can be improved, and the monitoring efficiency can be improved. Therefore, through the intelligent muck truck monitoring system based on artificial intelligence, accurate efficiency of muck transportation supervision is realized, an efficient and accurate muck transportation supervision mode is created, the digitization level of muck transportation supervision can be improved, and the level of a smart city and digitization treatment is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, smart city and digital governance, and particularly relates to a monitoring video target detection method and system based on artificial intelligence. BACKGROUND

[0002] The slag is a mixed waste produced by construction engineering, mainly including but not limited to excavated earthwork, construction waste, decoration waste, engineering silt, the slag truck is a truck specially used for transporting construction waste, engineering slag and other wastes, the slag is generally transported by the slag truck according to the relevant requirements from the slag producing place to the sink field, and then the sink field carries out the corresponding treatment, wherein, the transportation of the slag is generally transported by the construction unit (the producing party), the professional transportation enterprise (the carrier) and other responsible subjects, and in order to avoid the problems including but not limited to overloading, dust pollution, slag scattering, stealing and dumping, "black slag truck" and other problems in the process of slag transportation, the corresponding responsible subject needs to carry out the transportation supervision of the slag.

[0003] In the traditional technology, the supervision of the slag truck is generally based on manual inspection or through the monitoring video snapshot mode of the construction site vehicle exit, road checkpoint, sink field entrance and the like to supervise the slag truck.

[0004] However, the present inventors find that in the traditional technology, the supervision of the slag truck through the monitoring video snapshot mode is generally to find problems by checking the video by the relevant personnel or to take the image of the monitoring video snapshot as the evidence after the problem occurs, and the above modes cannot timely and automatically find the problems in the process of slag transportation.

[0005] Therefore, how to improve the precision and efficiency of the slag transportation supervision technical means becomes a problem to be solved in the field of smart city and digital governance. SUMMARY

[0006] The technical problem solved by the present application is to solve the problem of low precision and efficiency of the slag transportation supervision technical means.

[0007] To solve the above technical problems, the present application provides the following technical solutions: a monitoring video target detection method based on artificial intelligence, comprising: based on a preset first camera array, collecting a departure monitoring video of a muck car, the departure monitoring video comprising a departure main monitoring video; based on a pre-trained muck car detection model, detecting whether the departure main monitoring video contains a muck car; if the detection is yes, identifying the vehicle identification and vehicle type of the muck car, and constructing an initial information feature matrix of the muck car according to the departure monitoring video and based on a preset feature matrix construction method; determining a preset standard information feature matrix corresponding to the vehicle type; based on a preset matrix similarity judgment method, judging whether the initial information feature matrix is similar to the preset standard information feature matrix; if the judgment is yes, determining that the target muck car corresponding to the vehicle identification is normal.

[0008] The present application also provides a monitoring video target detection system based on artificial intelligence, comprising: a first acquisition module for collecting a departure monitoring video of a muck car based on a preset first camera array, the departure monitoring video comprising a departure main monitoring video; a first detection module for detecting whether the departure main monitoring video contains a muck car based on a pre-trained muck car detection model; a first identification module for identifying the vehicle identification and vehicle type of the muck car if the detection is yes, and constructing an initial information feature matrix of the muck car according to the departure monitoring video and based on a preset feature matrix construction method; a first determination module for determining a preset standard information feature matrix corresponding to the vehicle type; a first judgment module for judging whether the initial information feature matrix is similar to the preset standard information feature matrix based on a preset matrix similarity judgment method; and a first determination module for determining that the target muck car corresponding to the vehicle identification is normal if the judgment is yes.

[0009] The beneficial effects of the present application are as follows: by collecting monitoring videos of different angles based on a multi-camera array, the slag car in the monitoring video is automatically detected, and the information feature matrix of the slag car is constructed, and then the information feature matrix is used to detect the target of the slag car in the monitoring video, realize the slag car, based on the camera array, collect multi-angle videos of multiple cameras, so as to obtain videos of different angles of the slag car, so as to judge whether the slag car is cleaned and sealed from different angles, accordingly, the initial information feature matrix of the slag car is constructed, and then compared with the preset standard information feature matrix of the normal slag car of the same type, to detect the target of the slag car from multiple angles, so as to judge whether the slag car has the problem corresponding to the abnormal slag car such as overload, not cleaning, not sealing, etc., which can not only ensure the accuracy of the corresponding data, but also quickly identify different violations, and can not only improve the accuracy of slag transportation supervision, but also improve the supervision efficiency, so as to realize the automatic detection and identification of the slag car, the automatic abnormal target detection of the corresponding violation behavior through the intelligent slag car monitoring system based on artificial intelligence, realize the precise efficiency of slag transportation supervision, create an efficient and accurate slag transportation supervision mode, reduce the cost of manual checking, reduce the misjudgment rate, improve the digital level of slag transportation supervision based on artificial intelligence, and improve the level of smart city and digital governance. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 The overall concept and application environment of the monitoring video target detection method based on artificial intelligence provided by the embodiment of the present application are shown in the schematic diagram.

[0011] Figure 2 The flowchart of the monitoring video target detection method based on artificial intelligence provided by the embodiment of the present application is shown in the schematic diagram.

[0012] Figure 3 The first sub-flowchart of the monitoring video target detection method based on artificial intelligence provided by the embodiment of the present application is shown in the schematic diagram.

[0013] Figure 4 The second sub-flowchart of the monitoring video target detection method based on artificial intelligence provided by the embodiment of the present application is shown in the schematic diagram.

[0014] Figure 5 The schematic block diagram of the monitoring video target detection system based on artificial intelligence provided by the embodiment of the present application is shown in the schematic diagram. DETAILED DESCRIPTION

[0015] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all embodiments.

[0016] The embodiment of the present application provides a kind of based on artificial intelligence's monitoring video target detection method and system, the method and system can be applied in including but not limited to vehicle terminal, smart phone, tablet computer, server, cloud platform and so on Equipment, and in including but not limited to slag transport supervision business scene based on artificial intelligence's monitoring video target detection when being used.

[0017] The embodiment of the present application provides a kind of based on artificial intelligence's monitoring video target detection method and system, the method and system can be applied in including but not limited to vehicle terminal, smart phone, tablet computer, server, cloud platform and so on Equipment, and in including but not limited to slag transport supervision business scene based on artificial intelligence's monitoring video target detection when being used. Figure 1 The embodiment of the present application provides a kind of based on artificial intelligence's monitoring video target detection method and system, the method and system can be applied in including but not limited to vehicle terminal, smart phone, tablet computer, server, cloud platform and so on Equipment, and in including but not limited to slag transport supervision business scene based on artificial intelligence's monitoring video target detection when being used. Figure 1 Figure 1 The embodiment of the present application provides a kind of based on artificial intelligence's monitoring video target detection method and system, the method and system can be applied in including but not limited to vehicle terminal, smart phone, tablet computer, server, cloud platform and so on Equipment, and in including but not limited to slag transport supervision business scene based on artificial intelligence's monitoring video target detection when being used. Figure 1 ​As shown, the first camera array, the second camera array, and the road monitoring point at the front end respectively communicate with the muck truck transportation supervision background service end. The service end can receive the out-of-site monitoring video of the first camera array, the in-site monitoring video of the second camera array, the positioning information of the muck truck, and the video data of the road monitoring point, and construct an initial information feature matrix of the muck truck according to the out-of-site monitoring video of the first camera array to determine whether the muck truck is abnormal; monitor the real-time positioning of the muck truck according to the positioning information of the muck truck and the video data of the road monitoring point, and when the positioning is abnormal, determine whether the muck truck travels along the preset transportation route and whether the muck truck is abnormal in dust raising and throwing according to the video of the road monitoring point based on time sequence; determine whether the muck truck entering the disposal site is abnormal according to the similarity between the final information feature matrix corresponding to the in-site monitoring video and the initial information feature matrix of the out-of-site monitoring video, and the detection result of the abnormal muck truck can be sent to the relevant personnel such as the supervisor, the muck truck driver, and the muck truck transportation person in charge through the vehicle terminal and the client APP, so that the monitoring video of different angles is collected based on the multi-camera array to automatically detect the muck truck in the monitoring video, construct the information feature matrix of the muck truck, detect the abnormality of the muck truck in the monitoring video, and determine whether the muck truck is finally abnormal when entering the disposal site based on the similarity detection of the information feature matrix of the out-of-site and the information feature matrix of the in-site, and the three-stage detection of the whole operation process corresponding to the out-of-site, transportation, and in-site of the muck truck can be realized to solve the problem of low precision and efficiency of the supervision technology means caused by the simple snapshot of the muck truck in the traditional technology. Therefore, the intelligent muck truck monitoring system based on artificial intelligence can realize the precise efficiency of muck transportation supervision, improve the digital level of muck transportation supervision based on artificial intelligence, and thus improve the level of smart city and digital governance. The client can be but is not limited to various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the APP is the abbreviation of Application (application), which usually refers to a software program running on a smart phone, tablet computer, or other mobile device. The service end can be realized by a cloud platform, an independent server, or a server cluster composed of multiple servers. The present application will be described in detail through specific embodiments.

[0018] Embodiment 1, please refer to Figure 1 and Figure 2 , Figure 1 The overall concept and application environment diagram of the monitoring video target detection method based on artificial intelligence provided by the embodiment of the present application; Figure 2 The flowchart of the monitoring video target detection method based on artificial intelligence provided by the embodiment of the present application. As shown in Figure 2As shown, in this embodiment, the method includes but is not limited to the following steps S21-S27:

[0019] S21, based on the preset first camera array, collecting the out-of-site monitoring video of the muck truck, the out-of-site monitoring video including the out-of-site main monitoring video.

[0020] Explanatorily, the first camera array is preset, that is, the preset first camera array, which includes but is not limited to high-definition cameras, panoramic monitoring cameras, thermal imaging cameras, and dustproof and waterproof cameras, and the suitable cameras can be selected from the existing camera types according to the needs, and the preset first camera array can be deployed according to the needs. Figure 1 In the example, the preset first camera array includes a front-view camera 11, a right-side-view camera 12, a left-side-view camera 13, a top-view camera 14, and a rear-view camera 15, wherein the front-view camera 11 represents a monitoring video of the muck truck from the front view, and the corresponding front-view monitoring video is collected, the right-side-view camera 12 represents a monitoring video of the muck truck from the right side view, and the corresponding right-side-view monitoring video is collected, and the like, and the camera array refers to a system composed of multiple cameras according to a specific layout, which realizes the functions that a single camera cannot complete through cooperative work. The "first" in the first camera array is only used to distinguish different camera arrays, and is not used to limit the camera array. The other similar terms in the embodiment of the present application are used in the same way.

[0021] According to the above idea and deployment, based on the preset first camera array, the out-of-site monitoring video of the muck truck driving out of the vehicle checkpoint corresponding to the construction site of the muck source is collected, and the out-of-site monitoring video includes the out-of-site main monitoring video, wherein the main monitoring video refers to the monitoring video that can identify the muck truck in all out-of-site monitoring videos, and the main monitoring video can include but is not limited to the front-view monitoring video shot by the front-view camera and the rear-view monitoring video shot by the rear-view camera. The main monitoring video can be specified by relevant personnel according to experience, for example, the above-mentioned front-view monitoring video is specified as the main monitoring video, or it can be automatically learned from unlabeled data based on clustering and other unsupervised machine learning algorithms.

[0022] S22, based on the pre-trained muck truck detection model, detecting whether the out-of-site main monitoring video contains the muck truck.

[0023] Illustratively, the slag car detection model is preset and pre-trained, that is, a pre-trained slag car detection model is obtained, the pre-trained slag car detection model represents a model that is pre-trained and can be directly deployed in an application environment and targets slag car detection, the pre-trained slag car detection model includes but is not limited to a YOLO model, a Faster R-CNN, and a model based on a Transformer, and can also be selected from existing target detection models as needed.

[0024] According to the above concept and setting, based on the pre-trained slag car detection model, first, based on the out-of-scene monitoring video stream, it is detected whether the out-of-scene main monitoring video contains a slag car, that is, whether the vehicle portal corresponding to the slag car out-of-scene has a slag car out-of-scene.

[0025] S23, in the case where the out-of-scene main monitoring video contains a slag car, identifying the vehicle identification and model type of the slag car, and constructing an initial information feature matrix of the slag car according to the out-of-scene monitoring video and based on a preset feature matrix construction method.

[0026] Illustratively, the feature matrix construction method is preset, that is, the preset feature matrix construction method, the preset feature matrix construction method represents a method of constructing a plurality of features of a slag car into a matrix, the preset feature matrix construction method generally includes but is not limited to feature extraction of a slag car (feature extraction from an out-of-scene monitoring video), feature selection (screening of features with the most distinguishing and representative features), feature coding (numerical coding of category type features), feature standardization (unification of scales of different dimension features), matrix organization (organization of feature vectors into a matrix in the order of features such as time sequence or event sequence).

[0027] According to the above concept and setting, in the case that the on-site monitoring video contains the muck truck, the vehicle identification of the muck truck is automatically identified based on the preset vehicle identification identification method, the vehicle identification includes but is not limited to the license plate, and the vehicle identification is generally the license plate, and the vehicle type of the muck truck is automatically identified based on the preset vehicle type identification model, the vehicle type includes but is not limited to the U-shaped bucket compartment, the rectangular bucket compartment, the self-unloading compartment, the rear eight wheels (8x4 drive), the front four rear eight (6x2 drive), and the semi-trailer muck truck. The preset vehicle identification identification method represents a pre-set method for identifying the vehicle identification of the muck truck, and the preset vehicle identification identification method includes but is not limited to an image processing method (such as license plate positioning, character segmentation, character recognition, etc.), a deep learning-based method (such as a YOLO-LPR-based model, a DAN model (i.e. Deformable Attention Network)); the preset vehicle type identification model represents a pre-set model for identifying the vehicle type of the muck truck, and the preset vehicle type identification model includes but is not limited to a target detection model corresponding to a YOLO model or a PP-YOLOE model. It should be noted that for those skilled in the art, within their research and development capabilities and technical knowledge, it can be understood that vehicle identification and vehicle type identification can refer to existing technical means, which will not be described here, and vehicle identification and vehicle type identification can be independent modules or integrated into the pre-trained muck truck detection model, which is not limited here.

[0028] In addition, according to the on-site monitoring video, and based on the preset feature matrix construction method, such as feature extraction, feature screening, feature coding, feature standardization, matrix organization, etc. of the on-site monitoring video, an initial information feature matrix corresponding to the muck truck is constructed, the initial information feature matrix represents a feature set of the muck truck information when the muck truck drives out of the muck site vehicle checkpoint, and the "initial" in the initial information feature matrix is used to distinguish different information feature matrices, and is not used to limit different information feature matrices. Other similar terms in the embodiments of the present application are used in the same way, and the elements in the initial information feature matrix represent information related to the muck truck and the transportation of muck, including but not limited to the cleaning features and sealing features of the muck truck.

[0029] S24, determining a preset standard information feature matrix corresponding to the vehicle type.

[0030] Explanatorily, as described above, the vehicle type includes but is not limited to a U-shaped bucket compartment, a rectangular bucket compartment, a self-unloading compartment, a rear eight-wheel (8x4 drive), a front four rear eight (6x2 drive), a semi-trailer muck truck, for each vehicle type of the muck truck, the corresponding standard information feature matrix is preset, that is, the preset standard information feature matrix, the preset standard information feature matrix is corresponding to the initial information feature matrix, that is, the structure and elements of the preset standard information feature matrix are consistent with and corresponding to the initial information feature matrix, thus, the initial information feature matrix and the preset standard information feature matrix are comparable, and the preset standard information feature matrix represents the matrix of the corresponding features of the muck truck of the vehicle type under the normal state when transporting muck, and the elements in the preset standard information feature matrix represent the information related to the muck truck and the transportation of muck including but not limited to the cleaning feature and the sealing feature of the muck truck under the normal state when transporting muck. Since the vehicle type of each muck truck is different, the representation of the muck truck under the normal state when transporting muck is also different, and the representation is to convert abstract or complex entities (such as objects, concepts, and relationships) into symbolic forms that can be perceived, stored, or processed, thus, for each vehicle type of the muck truck, the corresponding preset standard information feature matrix is preset, and the initial information feature matrix of the muck truck of the vehicle type is compared with the corresponding preset standard information feature matrix to determine whether the current muck truck is abnormal, which can further improve the accuracy of the muck truck abnormality detection.

[0031] According to the above idea and setting, the muck truck is detected in the scene monitoring video, and the vehicle type of the muck truck is determined, the preset standard information feature matrix corresponding to the vehicle type is determined, and the standard for determining whether the muck truck detected in the scene monitoring video is abnormal is established.

[0032] S25, judging whether the initial information feature matrix is similar to the preset standard information feature matrix based on the preset matrix similarity judgment mode;

[0033] S26, in the case that the initial information feature matrix is similar to the preset standard information feature matrix, determining that the target muck truck corresponding to the vehicle identification is normal;

[0034] S27, in the case that the initial information feature matrix is not similar to the preset standard information feature matrix, determining that the target muck truck is abnormal.

[0035] Explanatorily, the matrix similarity judgment mode is preset, i.e., a preset matrix similarity judgment mode, which represents a mode of judging the similarity between different matrices. The preset matrix similarity judgment mode includes but is not limited to a distance measurement method (Euclidean distance, Manhattan distance, Mahalanobis distance, etc., which measures the difference between matrices. The greater the distance, the lower the similarity), a similarity coefficient method (cosine similarity, Pearson correlation coefficient, Jaccard similarity coefficient, etc., which calculates the similarity between matrices. The closer the value to 1, the more similar), and a matrix similarity classification based on deep learning (direct classification (end-to-end), similarity learning (Siamese Network), etc.).

[0036] According to the above concept and setting, and based on the preset matrix similarity judgment mode, it is judged whether the initial information feature matrix is similar to the preset standard information feature matrix. In the case where the initial information feature matrix is similar to the preset standard information feature matrix, it is indicated that the cleaning features, sealing features, etc. related to the transportation of the slag soil detected in the out-of-plant monitoring video are similar to the corresponding features of the slag soil vehicle of this type when the slag soil vehicle is in a normal state during the transportation of the slag soil, and it is determined that the target slag soil vehicle corresponding to the vehicle identification is normal, i.e., it is determined that the cleaning features, sealing features, etc. related to the transportation of the slag soil detected in the out-of-plant monitoring video are normal, and it is defaulted that the detected slag soil vehicle is normal. Similarly, in the case where the initial information feature matrix is not similar to the preset standard information feature matrix, it is determined that the target slag soil vehicle is abnormal, i.e., it is determined that the cleaning features, sealing features, etc. related to the transportation of the slag soil detected in the out-of-plant monitoring video are abnormal, and it is defaulted that the detected slag soil vehicle is normal. Thus, based on the first camera array, different angle monitoring videos are collected to automatically detect the target slag soil vehicle in the monitoring video, and the information feature matrix of the slag soil vehicle is constructed, and then the abnormality of the slag soil vehicle in the monitoring video is comprehensively detected based on the information feature matrix.

[0037] Further, based on the preset matrix similarity judgment mode, it is judged whether the initial information feature matrix is similar to the preset standard information feature matrix, including:

[0038] Based on the deep learning-based pre-training matrix similarity classification model, it is judged whether the initial information feature matrix is similar to the preset standard information feature matrix.

[0039] Specifically, a matrix similarity classification model based on deep learning is preset and corresponding pre-training is performed, that is, a pre-trained matrix similarity classification model based on deep learning is obtained, the pre-trained matrix similarity classification model represents a deep learning model that can be directly deployed in an application environment and targets matrix similarity classification, and the pre-trained matrix similarity classification model includes but is not limited to direct classification (end-to-end, such as a Transformer model, a convolutional neural network, and an MLP neural network (Multilayer Perceptron)), similarity learning (Siamese Network), and an autoencoder. The direct classification (end-to-end) is to splice (such as channel superposition or horizontal splicing) a matrix initial information feature matrix and a preset standard information feature matrix, input the pre-trained matrix similarity classification model based on a Transformer model, a convolutional neural network (CNN), or an MLP neural network, and output a similarity classification result (binary classification (normal / abnormal) or multi-classification (abnormal type)) between the matrix initial information feature matrix and the preset standard information feature matrix. The similarity learning (Siamese Network) is to calculate a similarity after a network extracts features, that is, for the Siamese neural network, the initial information feature matrix and the preset standard information feature matrix are input, and a similarity score between the two matrices is output. The autoencoder is trained by using normal samples (the preset standard information feature matrix),

[0040] When the initial information feature matrix is input, a reconstruction error is calculated, and if the error exceeds a threshold, it is determined to be abnormal. In addition, in the implementation of the technical scheme of the present application, the above-mentioned suitable model can be selected as needed.

[0041] According to the above concept and setting, the pre-trained matrix similarity classification model based on deep learning determines whether the initial information feature matrix is similar to the preset standard information feature matrix, which can fully utilize the automatic and adaptive characteristics of deep learning, automatically adapt the various feature structures of the slag car contained in the initial information feature matrix, such as cleaning features and sealing features, and at the same time, through data-driven optimization, automatically adapt the complex monitoring video scenes such as nonlinearity and high noise corresponding to the off-site monitoring video, which can further improve the accuracy of the slag car target detection in the monitoring video.

[0042] The embodiment of the present application collects monitoring videos of different angles based on a multi-camera array, automatically detects the slag car in the monitoring video, constructs an information feature matrix of the slag car, and then detects the target according to the information feature matrix. Abnormalities of the slag car in the monitoring video, realize the slag car when it leaves the scene, based on the camera array, collect multi-angle videos of multiple cameras, so as to obtain videos of the slag car from different angles, so as to judge whether the slag car is cleaned and sealed and other problems from different angles. Accordingly, the initial information feature matrix of the slag car is constructed, and the preset standard information feature matrix of the normal slag car of the vehicle type is compared to detect the target of the slag car from multiple angles. Abnormalities, so as to judge whether the slag car has problems corresponding to the abnormal slag car such as overloading, not cleaning, not sealing, etc. It can not only ensure accurate data, but also quickly identify different violations. Also, it can improve the accuracy of slag transportation supervision and improve the efficiency of supervision, so as to realize the automatic detection and identification of slag cars through the intelligent slag car monitoring system based on artificial intelligence, automatically detect the abnormal target corresponding to the violation behavior, and realize the precise efficiency of slag transportation supervision. Create an efficient and accurate slag transportation supervision mode, reduce manual checking costs, reduce the misjudgment rate, improve the digital level of slag transportation supervision based on artificial intelligence, and improve the level of smart city and digital governance.

[0043] In an embodiment, please refer to Figure 1 With Figure 3 , Figure 3 The first sub-flow diagram of the monitoring video target detection method based on artificial intelligence provided by the embodiment of the present application. As Figure 3 Indicated, in the embodiment, after determining that the target slag car corresponding to the vehicle identification has no abnormality, the method further comprises:

[0044] S31, monitoring the real-time positioning of the target slag car;

[0045] S32, if the positioning abnormality is detected, determining the position of the abnormality to obtain the abnormal position;

[0046] S33, determining the preset transportation route of the target slag car;

[0047] S34, according to the preset transportation route, screening out n continuous road monitoring points covering the abnormal position;

[0048] S35, extracting video data of the time period corresponding to the road monitoring point, and generating a video sequence arranged in time sequence to obtain a road monitoring point video sequence based on time sequence, n is a natural number;

[0049] S36, detecting whether the target dump truck appears in the video of each road monitoring point based on the pre-trained dump truck detection model and the road monitoring point video sequence;

[0050] S37, if the detection is yes, determining that the target dump truck is normal;

[0051] S38, if the detection is no, determining that the target dump truck is abnormal.

[0052] Explanatorily, please continue to refer to Figure 1 In the case that the target dump truck starts to transport the muck, the real-time positioning of the target dump truck is monitored. The real-time positioning can adopt, but is not limited to, GPS positioning (Global Positioning System), Beidou positioning, base station positioning, or a combination of the above manners. The real-time positioning manner can refer to the existing corresponding technical means, which will not be described here. Generally, the real-time positioning of the target dump truck is monitored to monitor the behaviors such as overspeed, illegal dumping, and route deviation, which is an efficient and accurate way.

[0053] In addition, whether the real-time positioning of the target dump truck is abnormal is monitored. When the real-time positioning of the target dump truck is normal, the target dump truck is monitored by real-time positioning. If it is detected that the positioning of the target dump truck is abnormal, the position of the abnormality is determined, and the abnormal position is obtained. The abnormal position represents the position where the real-time positioning of the dump truck is abnormal. Generally, it is the last position where the real-time positioning of the target dump truck can be detected, such as the abnormal position shown in the example of Figure 1

[0054] Since the transportation route of the dump truck is generally a pre-set fixed line, i.e., a pre-set transportation route, and the road monitoring points are pre-set on the pre-set transportation route, the road monitoring points represent the video monitoring points set on the road of the pre-set transportation route. The road monitoring points can borrow the video data of the public video monitoring system such as traffic electronic police, public security monitoring camera, intelligent lamp pole integrated camera, highway ETC gantry and roadside monitoring as far as possible. According to the pre-set transportation route of the target dump truck and the position of the road monitoring point and the abnormal position, n continuous road monitoring points covering the abnormal position are screened out, i.e., n continuous road monitoring points before and after the abnormal position are obtained, and the abnormal position is between the n continuous road monitoring points. The video data of the corresponding time period of the road monitoring point is extracted, i.e., the video data of the target dump truck passing through (the fact has occurred) or should pass through (predicted to pass through) the road monitoring point and before and after the road monitoring point is extracted. A video sequence arranged in time sequence is generated, and a road monitoring point video sequence based on time sequence is obtained. n is a natural number. The road monitoring point video sequence represents the driving track sequence of the target dump truck under normal circumstances. ​

[0055] According to the road monitoring point video sequence, and based on the pre-trained muck truck detection model, it is detected whether the target muck truck appears in the video of each road monitoring point, that is, whether the target muck truck appears in the frame-by-frame image corresponding to the monitoring video of each road monitoring point, until the target muck truck is detected or all frame images are detected without detecting the target muck truck, wherein the pre-trained muck truck detection model can use the same muck truck detection model described above. If the above detection is yes, it indicates that the target muck truck normally passed through the corresponding road monitoring point within the corresponding time, and it is determined that the target muck truck is normal. Similarly, if the above detection is no, it indicates that the target muck truck did not pass through the corresponding road monitoring point within the corresponding time, especially that the target muck truck was not detected in the video data of the road monitoring point after the abnormal position, and it is determined that the target muck truck is abnormal. The target muck truck has risks such as signal interruption, deviation from the electronic fence, abnormal speed, and dumping of muck, that is, the target muck truck has abnormal risks. Therefore, after the real-time positioning of the muck fails, video analysis, detection and tracking are automatically triggered, automatic switching between different detection modes of the muck truck is realized, the monitoring mode of the muck truck in transportation is dynamically switched, a collaborative and efficient detection effect is realized, video analysis, detection and tracking are performed based on the time sequence road monitoring point video sequence, continuous spatiotemporal tracking of the target muck truck is realized, and the accuracy of target muck truck detection is improved.

[0056] Further, detecting whether the target muck truck appears in the video of each road monitoring point comprises:

[0057] Determining a driving speed range of the target muck truck;

[0058] According to the driving speed range and the preset transportation route, a target time range of the target muck truck passing through each road monitoring point is determined;

[0059] Detecting whether the target muck truck appears in the video of each road monitoring point within the corresponding target time range.

[0060] Specifically, since the target muck truck will limit the maximum speed and the minimum speed, that is, the driving speed range of the target muck truck is determined, and thus the distance from the vehicle aperture of the muck truck at the place to each road monitoring point is determined, the target time range of the target muck truck passing through each road monitoring point can be calculated. The target time range does not need to be too accurate, and even can be extended on the basis of the calculated time range to obtain the target time range, so as to estimate the reasonable time range as much as possible on the basis of more adaptability to the complexity of driving, so that the detection of the target muck truck is adapted to the complex driving environment and the detection of the target muck truck is more accurate. Therefore, by limiting the target time range, it can be monitored whether the target muck truck appears in the video of each road monitoring point, so as to avoid the behaviors such as signal interruption, deviation from the electronic fence, abnormal speed, and dumping of muck of the muck truck, and further improve the precision and efficiency of the muck transportation supervision.

[0061] Further, the video data is video data of an external view angle behind the target muck truck, the pre-trained muck truck detection model comprises a muck truck recognition module and a sealing state analysis module; based on the pre-trained muck truck detection model, whether the target muck truck appears in the video of each road monitoring point is detected, comprising:

[0062] based on the muck truck recognition module, whether the target muck truck appears in the video of each road monitoring point is detected;

[0063] In the case where it is detected that the target muck truck appears in the video of the road monitoring point, based on the sealing state analysis module, whether the target muck truck is abnormal in sealing is detected, and the sealing abnormality includes at least one of the following: a sealing cover is not closed, muck spills, or a muck truck compartment structure is deformed.

[0064] Specifically, the video data is the video data of the external rear view of the target dump truck, that is, the video data is the road monitoring data collected from the external rear view of the target dump truck. Moreover, the pre-trained dump truck detection model comprises a dump truck recognition module and a sealing state analysis module, that is, the pre-trained dump truck detection model integrates the dump truck recognition module and the sealing state analysis module, and performs the following detection: the dump truck recognition module is used to determine whether the target dump truck appears in the video of a single road monitoring point based on a deep learning target detection algorithm, the dump truck recognition module is a dump truck target detection module based on a model including but not limited to a YOLO model, a Faster R-CNN, and a Transformer model; and the sealing state analysis module is used to detect whether the target dump truck has a sealing abnormality including but not limited to an unsealed cover, a dump soil overflow, and a dump truck compartment structure deformation based on an image semantic segmentation technology, the image semantic segmentation technology is to classify each pixel in an image into a predefined semantic category, thereby achieving fine understanding and segmentation of the image, and the image semantic segmentation includes but is not limited to a fully convolutional network, a U-Net network, a Transformer-based model, and a DeepLab semantic segmentation model, which can improve the accuracy of dump truck sealing detection.

[0065] According to the above concept and arrangement, when detecting whether the target dump truck appears in the video of each road monitoring point based on the pre-trained dump truck detection model, the following detection is performed: first, whether the target dump truck appears in the video of each road monitoring point is detected based on the dump truck recognition module; and when it is detected that the target dump truck appears in the video of the road monitoring point, whether the target dump truck has a sealing abnormality is detected based on the sealing state analysis module, the sealing abnormality includes at least one of an unsealed cover, a dump soil overflow, or a dump truck compartment structure deformation, thereby realizing monitoring of whether there is a problem of illegal driving of deviating from a preset transportation route during the transportation of the dump truck in combination with the positioning of the dump truck and the detection of the road monitoring point video based on the time sequence, and it is possible to detect whether the target dump truck has problems such as illegal dumping, dust pollution, and dump soil scattering, which not only can balance the efficiency and accuracy of dump truck detection, but also can balance the monitoring of whether there is a problem such as illegal dumping, dust pollution, and dump soil scattering during the transportation of the dump truck, thereby further improving the precision and efficiency of the dump transportation supervision.

[0066] The embodiment of the present application can realize the cooperative detection mechanism of the positioning of the target muck truck and the time sequence of the video by combining the road monitoring point video based on the time sequence to determine whether the muck truck travels according to the preset transportation route when the real-time positioning of the target muck truck appears signal shielding, artificial interference and other positioning abnormalities and failures, which can not only make full use of the efficient and convenient positioning to track and detect the motion trajectory of the target muck truck, but also automatically trigger the video analysis, detection and tracking after the positioning failure, so as to realize the necessary automatic switching between different detection modes of the muck truck, realize the synergistic detection effect by dynamically switching the monitoring mode of the muck truck in the transportation process, solve the two pain points of the "monitoring blank after positioning failure" and "low efficiency of pure video analysis" in the prior art, and realize the "positioning abnormality triggering mechanism + space-time video data fusion", so as to fully utilize the efficiency of positioning tracking and the accuracy of video monitoring based on the time sequence, so as to improve the monitoring accuracy and efficiency of the muck transportation, and improve the precision of the muck transportation monitoring.

[0067] In an embodiment, please refer to Figure 1 and Figure 4 , Figure 4 The second sub-process schematic diagram of the monitoring video target detection method based on artificial intelligence provided by the embodiment of the present application is shown in FIG. 6. Figure 4 As shown in the embodiment, after determining that the target muck truck corresponding to the vehicle identification is normal, the method further includes:

[0068] S41, based on a preset second camera array, collecting entry monitoring video of the muck truck entering the disposal field;

[0069] S42, detecting the target muck truck according to the entry monitoring video;

[0070] S43, in the case of detecting the target muck truck, constructing a final information feature matrix corresponding to the initial information feature matrix according to the entry monitoring video;

[0071] S44, judging whether the final information feature matrix is similar to the initial information feature matrix based on the pre-trained matrix similarity classification model;

[0072] S45, in the case that the final information feature matrix is similar to the initial information feature matrix, determining that the target muck truck is normal;

[0073] S46, in the case that the final information feature matrix is not similar to the initial information feature matrix, determining that the target muck truck is abnormal.

[0074] By the way, referring to the construction of the initial information feature matrix described above, the difference between the initial information feature matrix and the final information feature matrix is that the initial information feature matrix corresponds to the out-of-site monitoring video of the vehicle portal of the muck truck leaving the site, and the final information feature matrix corresponds to the in-of-site monitoring video of the vehicle portal of the muck truck entering the disposal site. Thus, based on the preset second camera array, the in-of-site monitoring video of the vehicle portal of the muck truck entering the disposal site is collected; in the case of detecting the target muck truck, the final information feature matrix corresponding to the initial information feature matrix is constructed according to the in-of-site monitoring video, wherein the "final" in the final information feature matrix corresponds to the "initial" in the initial information feature matrix, and the two are antonyms, which can be understood accordingly; then, whether the final information feature matrix is similar to the initial information feature matrix is judged based on the pre-trained matrix similarity classification model; in the case that the final information feature matrix is similar to the initial information feature matrix, it is determined that the target muck truck is not abnormal; in the case that the final information feature matrix is not similar to the initial information feature matrix, it is determined that the target muck truck is abnormal, thus realizing the final detection of the muck truck entering the disposal site, and realizing the similarity detection based on the information feature matrix of the muck truck leaving the site and the information feature matrix of the muck truck entering the disposal site to determine whether the muck truck entering the disposal site is finally abnormal, realizing the two-end detection of the "out-of-site" and "in-of-site" of the muck truck, and realizing the whole-process online detection of the whole operation process corresponding to the "out-of-site-transportation-in-of-site" of the muck truck from the construction site to the disposal site.

[0075] In the embodiment of the present application, whether the final information feature matrix corresponding to the target muck truck entering the disposal site is consistent with the initial information feature matrix is detected to determine whether the target muck truck is abnormal, and in the case that the final information feature matrix is consistent with the initial information feature matrix, it is detected that the target muck truck is not abnormal, otherwise, it is detected that the target muck truck is abnormal, realizing the comparison of the multi-dimensional information of the muck truck based on the two ends of the "out-of-site" and "in-of-site" to check whether the muck truck is abnormal, and realizing the three-section detection of the whole operation process corresponding to the "out-of-site-transportation-in-of-site" of the muck truck, which can not only ensure the accuracy of the corresponding data, but also quickly identify different irregular behaviors, and can not only improve the accuracy of the muck transportation supervision, but also improve the supervision efficiency, and can realize the whole-process seamless compact supervision of the whole operation process of the muck truck from the out-of-site to the in-of-site, so as to realize the precise efficiency of the muck transportation supervision based on the intelligent muck truck monitoring system based on artificial intelligence, create an efficient and precise muck transportation supervision mode, reduce the cost of manual checking, reduce the misjudgment rate, improve the digital level of the muck transportation supervision based on artificial intelligence, and thus improve the level of smart city and digital governance.

[0076] In an embodiment, the off-site monitoring video comprises at least two of the following perspectives: front perspective monitoring video of the muck truck, side perspective monitoring video, rear perspective monitoring video, top perspective monitoring video; according to the off-site monitoring video, and based on a preset feature matrix construction manner, an initial information feature matrix of the muck truck is constructed, including:

[0077] According to the perspective video included in the off-site monitoring video, the following information features of the corresponding perspective video are extracted:

[0078] Based on the pre-trained video feature extraction model of deep learning, the information features of the front perspective monitoring video are extracted to obtain initial front perspective information features;

[0079] Based on the pre-trained video feature extraction model, the information features of the side perspective monitoring video are extracted to obtain initial side perspective information features;

[0080] Based on the pre-trained video feature extraction model, the information features of the rear perspective monitoring video are extracted to obtain initial rear perspective information features;

[0081] Based on the pre-trained video feature extraction model, the information features of the top perspective monitoring video are extracted to obtain initial top perspective information features;

[0082] At least two of the above extracted initial front perspective information features, initial side perspective information features, initial rear perspective information features, and initial top perspective information features are combined into a matrix according to a preset perspective order to obtain an initial information feature matrix corresponding to the muck truck.

[0083] Explanatorily, a video feature extraction model is pre-set and pre-trained, i.e., a pre-trained video feature extraction model is obtained, which represents a pre-trained model that can be directly deployed in an application environment and aims at the feature extraction of the above monitoring video. The pre-trained video feature extraction model includes but is not limited to YOLO model, Faster R-CNN, and Transformer-based model, and can also be selected from existing target detection models according to needs, wherein the features of the monitoring video include but are not limited to cleaning features, sealing features, spilling and dripping features, and overload features (such as large-volume construction waste extending out of the truck cabin).

[0084] According to the above concept and setting, the off-site monitoring video comprises at least two of the following perspectives: front perspective monitoring video of the muck truck, side perspective monitoring video, rear perspective monitoring video, top perspective monitoring video, and the monitoring video of each perspective is as described above.

[0085] Then, when constructing the initial information feature matrix corresponding to the slag car, the corresponding exit monitoring video contains the visual angle video, that is, the exit monitoring video includes the front visual angle monitoring video, and the following information feature extraction of the corresponding front visual angle monitoring video is performed. The exit monitoring video includes the front visual angle monitoring video and the rear visual angle monitoring video, and the following information feature extraction of the corresponding front visual angle monitoring video and the information feature extraction of the rear visual angle monitoring video are performed respectively. The other is similar, and thus at least two of the following information feature extraction of the visual angle video is performed: the pre-trained video feature extraction model based on deep learning is used to extract the information feature of the front visual angle monitoring video to obtain the initial front visual angle information feature. The initial front visual angle information feature represents the slag car feature set from the front visual angle of the slag car when the slag car exits the slag site vehicle clamp, and "initial" in the initial front visual angle information feature is used to distinguish different front visual angle information features and does not limit different front visual angle information features. Other similar terms in the embodiments of the present application are similar.

[0086] Finally, at least two of the above extracted initial front visual angle information feature, initial side visual angle information feature, initial rear visual angle information feature and initial top visual angle information feature are combined into a matrix according to a preset visual angle sequence to obtain the initial information feature matrix corresponding to the slag car, and the preset visual angle sequence is preset.

[0087] In the embodiments of the present application, the videos of the slag car from different visual angles are collected, and the corresponding information features are extracted to construct the initial information feature matrix corresponding to the slag car, and then the matrix form is used to judge whether the slag car is cleaned and sealed from different angles, so as to perform multi-angle target abnormal detection on the exit slag car to judge whether the slag car has problems corresponding to the slag car abnormalities such as overloading, not cleaning and not sealing. The corresponding data can be ensured to be accurate and reliable, different illegal behaviors can be quickly identified, the supervision accuracy of slag transportation can be improved, and the supervision efficiency can be improved, so that the intelligent slag car monitoring system based on artificial intelligence is used to realize automatic detection and identification of the slag car, automatic abnormal target detection of illegal behaviors, to realize the precise efficiency of slag transportation supervision, and to create an efficient and precise slag transportation supervision mode.

[0088] In an embodiment, at least two of the extracted initial front view information features, initial side view information features, initial rear view information features, and initial top view information features are arranged in a matrix according to a preset view order to obtain an initial information feature matrix corresponding to the dump truck.

[0089] Determine the actual load of the dump truck.

[0090] At least two of the extracted initial front view information features, initial side view information features, initial rear view information features, and initial top view information features are arranged in a matrix according to a preset view order and a preset view load order to obtain an initial information feature matrix corresponding to the dump truck.

[0091] Explanatorily, the actual load of the dump truck is determined, and the actual load of the dump truck can be obtained by using techniques including but not limited to direct weighing technology (installing a weighing sensor on the vehicle chassis to measure the load in real time), pressure sensor (monitoring the pressure change of the hydraulic suspension system to calculate the load), and strain gauge technology (installing a strain gauge on the key parts of the vehicle frame to measure the deformation to calculate the load), which can be derived from existing technical means, and will not be described here.

[0092] At least two of the extracted initial front view information features, initial side view information features, initial rear view information features, and initial top view information features are arranged in a matrix according to a preset view order and a preset view load order, that is, the view information features and the load features corresponding to the actual load are arranged in a matrix according to a certain order to obtain an initial information feature matrix corresponding to the dump truck, so that the actual load of the dump truck is taken as an element of the initial information feature matrix and is constituted, so as to comprehensively detect the abnormal target of the dump truck by combining the actual load of the dump truck with the video features, which can improve the efficiency of the abnormal target detection of the dump truck.

[0093] In the embodiment of the present application, the video features and the actual load features of the dump truck are arranged in an information feature matrix, and the information feature matrix is used to detect the abnormal target of the dump truck in the monitoring video, so as to comprehensively detect whether the dump truck has problems such as overload, no cleaning, and no sealing, and to realize automatic detection and identification of the vehicle target and the abnormal target of the dump truck in the monitoring video by using an intelligent dump truck monitoring system based on artificial intelligence, so as to realize precise efficiency of dump transportation supervision, create an efficient and precise dump transportation supervision mode, improve the digital level of dump transportation supervision based on artificial intelligence, and improve the level of smart city and digital governance.

[0094] In an embodiment, the preset standard information feature matrix includes a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix, and the elements in the preset standard information feature upper limit matrix, the preset standard information feature lower limit matrix and the initial information feature matrix are all quantified features; based on a preset matrix similarity determination manner, it is determined whether the initial information feature matrix is similar to the preset standard information feature matrix, including:

[0095] element-by-element comparison of the initial information feature matrix with the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix is performed;

[0096] it is determined whether each element in the initial information feature matrix falls within an upper and lower limit interval composed of a corresponding upper limit and a lower limit;

[0097] if the determination is no, it is determined that the initial information feature matrix is not similar to the preset standard information feature matrix;

[0098] if the determination is yes, it is determined that the initial information feature matrix is similar to the preset standard information feature matrix.

[0099] Explanatorily, the abnormal targets that can be detected by the monitoring video of the slag car include but are not limited to over-limit (the height of the slag pile exceeds the car compartment baffle or blocks the line of sight), poor sealing (non-closed transportation, such as tarpaulin not covered, too large car compartment gap), car body deformation (car compartment side inclination, depression), license plate shielding or contamination, no washing (muddy road), the features corresponding to the above-mentioned abnormalities can be quantified and the corresponding upper and lower limits can be determined, the quantified feature represents a feature that can be represented by a numerical value (discrete or continuous), which is usually standardized data extracted from original data through measurement, calculation or statistical method, which is used to objectively describe the attributes of the object and supports mathematical operation, comparison or modeling. The specific quantified features and upper and lower limits are shown in the following Table 1:

[0100] Table 1

[0101] Abnormal performance of dump truck Quantitative features Overrun Stacking height exceeds the proportion Tarpaulin not covered Tarpaulin coverage ratio (0-1) Carriage gap too large Gap width (cm) Carriage side tilt, depression Symmetry score (0-1) License plate blocking or staining License plate recognition confidence (0-1) Not cleaned Vehicle body soil coverage

[0102] Therefore, on the basis of the abnormal performance of the slag car, the upper and lower limits of the quantified features can be determined, and whether the slag car is abnormal can be determined through the upper and lower limits, and for the case where only the upper limit or the lower limit exists, the corresponding lower limit or upper limit can be filled with 0, for example, the lower limit of the quantified feature "height" of over-limit can be set to "0".

[0103] Among them, 1) the stack height exceeds the proportion, defined as "(actual stack height - carriage baffle height) / carriage baffle height", the detection method includes but is not limited to the following ways: based on image segmentation, identifying the baffle edge and the muck contour, calculating the exceeding proportion; 2) the tarpaulin coverage ratio, defined as the ratio of the area effectively covered by the tarpaulin on the top of the muck truck to the total area of the top of the truck, usually expressed in percentage (0%~100%), which can be detected based on image segmentation method, that is, by shooting the top view of the truck through the on-board camera, and through image semantic segmentation, such as using deep learning model (such as U-Net, DeepLab) to segment the tarpaulin area and truck area in the image, output the mask image of tarpaulin pixels (green) and uncovered pixels (red), and then calculate the proportion of tarpaulin pixels to total truck top pixels; 3) gap width, defined as the maximum physical gap distance between the tarpaulin and the truck baffle, the edge of the body, usually in centimeters (cm) or pixels (need to be calibrated), which can be detected based on deep learning end-to-end regression, for example, select the model "lightweight CNN (such as MobileNet) + regression head", and train the model "input truck local image, output gap width (unit: cm)", then deploy it to the application environment, input the collected current truck local image, and perform real-time prediction to directly output the gap width value; 4) symmetry score (0~1), defined as the mirror symmetry of the left and right key points (such as corner points, edges) of the truck, 1 means complete symmetry, and for key point detection, YOLO or OpenCV can be used to extract the geometric features of the truck; 5) license plate recognition confidence, defined as the recognition confidence of the OCR model for the license plate characters (such as EasyOCR output probability), the detection method is to use the OCR model to recognize the license plate in real time and output the confidence; 6) vehicle body mud coverage rate, defined as the pixel coverage rate of the vehicle body surface covered by mud, the detection method can be based on image semantic segmentation model to identify the vehicle body mud area (such as U-Net).

[0104] It should be noted that the above stack height exceeds the proportion, tarpaulin coverage ratio, gap width, symmetry score, inclination angle, local indentation depth, license plate recognition confidence, and vehicle body mud coverage rate correspond to the quantitative features and their detection methods, which are only exemplary descriptions. For those skilled in the art, with their research and development capabilities and technical knowledge, it can be understood that the abnormal performance of the muck truck includes but is not limited to the above-mentioned exemplary quantitative features and their respective detection methods, and other quantitative features or detection methods can be used accordingly, which are not limited herein.

[0105] According to the above description, the preset standard information feature matrix includes a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix, the preset standard information feature upper limit matrix represents a matrix composed of upper limits of corresponding quantitative features of the muck car in the corresponding vehicle type in a non-abnormal state during transportation of muck, and the elements of the preset standard information feature upper limit matrix represent upper limits of quantitative features after feature quantization of information related to transportation of muck of the muck car corresponding to the cleaning feature and the sealing feature of the muck car in a non-abnormal state during transportation of muck, and the preset standard information feature lower limit matrix represents a matrix composed of lower limits of corresponding quantitative features of the muck car in the corresponding vehicle type in a non-abnormal state during transportation of muck, and the elements of the preset standard information feature lower limit matrix represent lower limits of quantitative features after feature quantization of information related to transportation of muck of the muck car corresponding to the cleaning feature and the sealing feature of the muck car in a non-abnormal state during transportation of muck. Moreover, the elements of the preset standard information feature upper limit matrix, the preset standard information feature lower limit matrix and the initial information feature matrix are quantitative features.

[0106] According to the above idea and setting, when judging whether the initial information feature matrix is similar to the preset standard information feature matrix based on the preset matrix similarity judgment mode, the following steps are performed:

[0107] The initial information feature matrix is compared with the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix element by element, it is judged whether each element in the initial information feature matrix falls into an upper and lower limit interval composed of a corresponding upper limit and a lower limit, if the above judgment is no, it is determined that the initial information feature matrix is not similar to the preset standard information feature matrix, and if the above judgment is yes, it is determined that the initial information feature matrix is similar to the preset standard information feature matrix.

[0108] In the embodiment of the present application, by setting the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix, and comparing the elements in the initial information feature matrix with the elements in the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix respectively, it is judged whether each element in the initial information feature matrix falls into an upper and lower limit interval composed of a corresponding upper limit and a lower limit, so that the elements in the initial information feature matrix are controlled in the upper and lower limit interval composed of the corresponding elements in the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix, the flexibility and accuracy of comprehensive detection of the muck car can be realized by quantizing the corresponding features, so that the vehicle target and the muck car abnormal target in the monitoring video are automatically and comprehensively detected and recognized by the intelligent muck car monitoring system based on artificial intelligence, the precision of muck transportation supervision is realized, an efficient and precise muck transportation supervision mode is created, the digital level of muck transportation supervision based on artificial intelligence is improved, and the level of smart city and digital governance is improved.

[0109] In an embodiment, after determining that the target dump truck corresponding to the vehicle identification is abnormal, the method further comprises:

[0110] sending information of the abnormality of the target dump truck to a preset terminal, so that relevant personnel can know the abnormality of the target dump truck in time, the preset terminal comprising at least one of a vehicle terminal and an APP, and the relevant personnel comprising at least one of a supervisor, a dump truck driver and a dump truck transport person in charge.

[0111] Illustratively, after determining that the target dump truck corresponding to the vehicle identification is abnormal, the information of the abnormality of the target dump truck is sent to a preset terminal, the preset terminal comprising at least one of a vehicle terminal and a client APP, and the relevant personnel comprising at least one of a supervisor, a dump truck driver and a dump truck transport person in charge, so that the relevant personnel can know the abnormality of the target dump truck in time, and adverse consequences caused by illegal behaviors such as overloading, dust pollution, dump scattering and illegal dumping can be avoided in time.

[0112] In the embodiment of the application, the information of the abnormality of the target dump truck is sent to a preset terminal in time, so that the relevant personnel can know the abnormality of the target dump truck in time, which not only can timely inform the abnormality of the dump truck, but also can warn illegal behaviors such as overloading, dust pollution, dump scattering and illegal dumping, and prevent problems as much as possible, so as to achieve the purpose of preventing dump transportation from being illegal, thereby improving the accuracy of dump transportation supervision and the efficiency of supervision, so as to realize the precision and efficiency of dump transportation supervision through the intelligent dump truck monitoring system based on artificial intelligence, improve the digital level of dump transportation supervision based on artificial intelligence, and improve the level of smart city and digital governance.

[0113] It should be noted that the monitoring video target detection method based on artificial intelligence described in each of the above embodiments can combine the technical features contained in different embodiments as needed to obtain a combined embodiment, but all within the scope of protection required by the application.

[0114] In an embodiment, a monitoring video target detection system based on artificial intelligence is provided, which corresponds one-to-one to the monitoring video target detection method based on artificial intelligence in the above embodiments. Please refer to Figure 5 , Figure 5 The schematic block diagram of the monitoring video target detection system based on artificial intelligence provided for the embodiment of the application is shown in FIG. 1. Figure 5As shown, the artificial intelligence-based monitoring video target detection system 50 includes a first acquisition module 51, a first detection module 52, a first identification module 53, a first determination module 54, a first judgment module 55, and a first determination module 56. The functions of the above-mentioned modules are described in detail as follows: The first acquisition module 51 is configured to acquire a departure monitoring video of a muck truck based on a preset first camera array. The departure monitoring video includes a departure main monitoring video. The first detection module 52 is configured to detect whether the departure main monitoring video contains a muck truck based on a pre-trained muck truck detection model. The first identification module 53 is configured to identify a vehicle identification and a vehicle type of the muck truck if the detection is yes, and to construct an initial information feature matrix of the muck truck according to the departure monitoring video and based on a preset feature matrix construction method. The first determination module 54 is configured to determine a preset standard information feature matrix corresponding to the vehicle type. The first judgment module 55 is configured to judge whether the initial information feature matrix is similar to the preset standard information feature matrix based on a preset matrix similarity judgment method. The first determination module 56 is configured to determine that a target muck truck corresponding to the vehicle identification is normal if the judgment is yes.

[0115] In an embodiment, the monitoring video target detection system 50 further includes a first monitoring module configured to monitor real-time positioning of the target muck truck, a second determination module configured to determine a location of an abnormality if an abnormal positioning is detected, to obtain an abnormal location, a third determination module configured to determine a preset transportation route of the target muck truck, a first screening module configured to screen n continuous road monitoring points covering the abnormal location according to the preset transportation route, a first generation module configured to extract video data of a time period corresponding to the road monitoring points and generate a video sequence arranged in chronological order to obtain a road monitoring point video sequence based on a time sequence, n being a natural number, a second detection module configured to detect whether the target muck truck appears in a video of each of the road monitoring points according to the road monitoring point video sequence and based on a pre-trained muck truck detection model, and a second determination module configured to determine that the target muck truck is normal if the detection is yes.

[0116] In an embodiment, the second detection module includes a first determination submodule configured to determine a driving speed range of the target muck truck, a second determination submodule configured to determine a target time range of the target muck truck passing through each of the road monitoring points according to the driving speed range and the preset transportation route, and a first detection submodule configured to detect whether the target muck truck appears in a video of each of the road monitoring points within the corresponding target time range.

[0117] In an embodiment, the video data is video data of an external rear view of the target dump truck, the pre-trained dump truck detection model comprises a dump truck recognition module and a sealing state analysis module; the second detection module comprises: a second detection submodule, configured to detect whether the target dump truck appears in the video of each of the road monitoring points based on the dump truck recognition module; and a third detection submodule, configured to detect whether the target dump truck is abnormal in sealing based on the sealing state analysis module in a case where the target dump truck is detected to appear in the video of the road monitoring point, the abnormality in sealing including at least one of an un-closed sealing cover, spillage of dump, or deformation of a dump truck compartment structure.

[0118] In an embodiment, the monitoring video target detection system 50 further comprises: a second acquisition module, configured to acquire entry monitoring videos of dump trucks entering a disposal site based on a preset second camera array; a third detection module, configured to detect the target dump truck based on the entry monitoring videos; a first construction module, configured to construct a final information feature matrix corresponding to the initial information feature matrix based on the entry monitoring videos in a case where the target dump truck is detected; a second judgment module, configured to judge whether the final information feature matrix is similar to the initial information feature matrix based on the pre-trained matrix similarity classification model; and a third determination module, configured to determine that the target dump truck is not abnormal in a case where the final information feature matrix is similar to the initial information feature matrix.

[0119] In an embodiment, the exit monitoring video comprises at least two of the following perspectives: a front view of the dump truck, a side view, a rear view, and a top view; and the first recognition module 53 comprises:

[0120] a first extraction submodule, configured to extract information features of the corresponding perspective videos based on the perspective videos included in the exit monitoring video, including: extracting information features of the front view monitoring video based on a pre-trained video feature extraction model based on deep learning to obtain initial front view information features; extracting information features of the side view monitoring video based on the pre-trained video feature extraction model to obtain initial side view information features; extracting information features of the rear view monitoring video based on the pre-trained video feature extraction model to obtain initial rear view information features; and extracting information features of the top view monitoring video based on the pre-trained video feature extraction model to obtain initial top view information features;

[0121] The first constituent sub-module is configured to group at least two of the extracted initial front-view information features, initial side-view information features, initial rear-view information features, and initial top-view information features into a matrix according to a preset view order to obtain an initial information feature matrix corresponding to the dump truck.

[0122] In an embodiment, the first constituent sub-module includes a second determination sub-module configured to determine an actual load of the dump truck, and a second constituent sub-module configured to group at least two of the extracted initial front-view information features, initial side-view information features, initial rear-view information features, and initial top-view information features with the actual load into a matrix according to a preset view order and a preset load order to obtain the initial information feature matrix corresponding to the dump truck.

[0123] In an embodiment, the preset standard information feature matrix includes a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix, and elements in the preset standard information feature upper limit matrix, the preset standard information feature lower limit matrix, and the initial information feature matrix are all quantized features; the first judging module 55 includes a first comparison sub-module configured to compare the initial information feature matrix with the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix element by element, a first judgment sub-module configured to determine whether each element in the initial information feature matrix falls within an upper limit and a lower limit interval formed by the corresponding upper limit and lower limit, and a first determination sub-module configured to determine that the initial information feature matrix is not similar to the preset standard information feature matrix if the determination is negative.

[0124] In an embodiment, the first judging module 55 is specifically configured to use a pre-trained matrix similarity classification model based on deep learning to determine whether the initial information feature matrix is similar to the preset standard information feature matrix.

[0125] The embodiment of the present application provides a monitoring video target detection system based on artificial intelligence, which automatically detects a dump truck in a monitoring video by collecting monitoring videos with different views based on a multi-camera array, constructs an information feature matrix of the dump truck, and then detects an abnormality of the dump truck in the monitoring video according to the information feature matrix, thereby improving the accuracy and efficiency of the supervision of the dump truck transportation, and realizing the automatic detection and identification of the dump truck, the automatic abnormal target detection corresponding to the illegal behavior, and the precise efficiency of the supervision of the dump truck transportation through the intelligent dump truck monitoring system based on artificial intelligence.

[0126] The specific limitations of the artificial intelligence-based monitoring video target detection system can refer to the limitations of the artificial intelligence-based monitoring video target detection method described above, which will not be repeated here. Each module in the artificial intelligence-based monitoring video target detection system described above can be implemented by software, hardware, and combinations thereof, in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor calls and executes the operations corresponding to each of the above modules.

[0127] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media having computer-usable program code embodied therein. The storage media can be realized by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. These computer program instructions can also be stored in a computer readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable memory produce a product including instruction means, which realize the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0128] The non-company software tools or components appearing in the embodiments of the present application are only examples and do not represent actual use.

[0129] The related data collection in the embodiments of the present application meets the requirements of relevant laws and regulations, such as the Personal Information Protection Law of China, the GDPR (General Data Protection Regulation of the European Union) or other national and regional information security standards.

[0130] The related data collection in the embodiments of the present application meets the requirements of relevant laws and regulations, such as GDPR (General Data Protection Regulation of the European Union) or information security standards of other countries and regions.

[0131] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A surveillance video target detection method based on artificial intelligence, characterized in that: include: Based on a preset first camera array, an exit monitoring video of the muck truck is collected, wherein the exit monitoring video includes an exit main monitoring video; Based on the pre-trained muck truck detection model, detecting whether the main monitoring video of the exit contains a muck truck; If the above detection result is yes, identify the vehicle identification and vehicle type of the muck truck, and construct an initial information feature matrix of the muck truck based on the exit monitoring video and a preset feature matrix construction method; Determining a preset standard information feature matrix corresponding to the vehicle type; Based on a preset matrix similarity judgment method, judging whether the initial information feature matrix is ​​similar to the preset standard information feature matrix; If the above judgment is yes, it is determined that there is no abnormality in the target dump truck corresponding to the vehicle identification.

2. The method for detecting target in surveillance video based on artificial intelligence according to claim 1, wherein: After determining that the target muck truck corresponding to the vehicle identification has no abnormality, the method further includes: Monitoring the real-time positioning of the target muck truck; If a positioning anomaly is detected, the location where the anomaly occurs is determined and the anomaly position is obtained; Determining a preset transport route for the target muck truck; According to the preset transport route, n consecutive road monitoring points covering the abnormal location are screened out; Extracting video data of a time period corresponding to the road monitoring point and generating a video sequence arranged in chronological order to obtain a road monitoring point video sequence based on a time series, where n is a natural number; According to the video sequence of the road monitoring points and based on a pre-trained muck truck detection model, detecting whether the target muck truck appears in the video of each of the road monitoring points; If the above detection result is yes, it is determined that there is no abnormality in the target dump truck.

3. The method for detecting target in surveillance video based on artificial intelligence according to claim 2, wherein: Detecting whether the target dump truck appears in the video of each road monitoring point includes: Determining a driving speed range of the target muck truck; Determining a target time range for the target muck truck to pass through each of the road monitoring points based on the driving speed range and the preset transport route; Detect whether the target muck truck appears in the video of each road monitoring point within the corresponding target time range.

4. The method for detecting target in surveillance video based on artificial intelligence according to claim 2, wherein: The video data is video data from an external perspective behind the target muck truck, and the pre-trained muck truck detection model includes a muck truck recognition module and a sealing state analysis module; Based on the pre-trained muck truck detection model, detecting whether the target muck truck appears in the video of each road monitoring point includes: Based on the muck truck identification module, detecting whether the target muck truck appears in the video of each road monitoring point; When the target dump truck is detected to appear in the video of the road monitoring point, based on the sealing status analysis module, it is detected whether the target dump truck has a sealing abnormality, and the sealing abnormality includes at least one of the sealing cover not being closed, the dump truck overflowing, or the dump truck compartment structure being deformed.

5. The method for detecting target in surveillance video based on artificial intelligence according to claim 1, wherein: After determining that the target muck truck corresponding to the vehicle identification has no abnormality, the method further includes: Based on the preset second camera array, capture the entry surveillance video of the muck truck entering the disposal site; Detecting the target muck truck based on the entry monitoring video; When the target muck truck is detected, constructing a final information feature matrix corresponding to the initial information feature matrix based on the entry monitoring video; Based on the pre-trained matrix similarity classification model, determining whether the final information feature matrix is ​​similar to the initial information feature matrix; When the final information feature matrix is ​​similar to the initial information feature matrix, it is determined that there is no abnormality in the target muck truck.

6. The method for detecting target in surveillance video based on artificial intelligence according to any one of claims 1 to 5, characterized in that: The exit monitoring video includes at least two of the following perspectives: a front perspective monitoring video of the muck truck, a side perspective monitoring video, a rear perspective monitoring video, and a top perspective monitoring video; According to the exit monitoring video and based on a preset feature matrix construction method, an initial information feature matrix of the muck truck is constructed, including: According to the view angle video included in the exit monitoring video, the following information feature extraction of the corresponding view angle video is performed: A pre-trained video feature extraction model based on deep learning is used to extract information features of the front view monitoring video to obtain initial front view information features; Extracting information features of the side view monitoring video based on the pre-trained video feature extraction model to obtain initial side view information features; Extracting information features of the rear-view monitoring video based on the pre-trained video feature extraction model to obtain initial rear-view information features; Extracting information features of the top-view monitoring video based on the pre-trained video feature extraction model to obtain initial top-view information features; At least two of the initial front perspective information features, initial side perspective information features, initial rear perspective information features, and initial top perspective information features extracted above are combined into a matrix according to a preset perspective order to obtain an initial information feature matrix corresponding to the dump truck.

7. The method for detecting target in surveillance video based on artificial intelligence according to claim 6, wherein: At least two of the extracted initial front perspective information features, initial side perspective information features, initial rear perspective information features, and initial top perspective information features are combined into a matrix according to a preset perspective order to obtain an initial information feature matrix corresponding to the muck truck, including: Determining the actual load of the muck truck; At least two of the initial front perspective information features, initial side perspective information features, initial rear perspective information features, and initial top perspective information features extracted above and the actual load are combined into a matrix according to a preset perspective order and a preset perspective load order to obtain an initial information feature matrix corresponding to the dump truck.

8. The method for detecting target in surveillance video based on artificial intelligence according to claim 1, wherein: The preset standard information feature matrix includes a preset standard information feature upper limit matrix and a preset standard information feature lower limit matrix, and the elements in the preset standard information feature upper limit matrix, the preset standard information feature lower limit matrix and the initial information feature matrix are all quantitative features; Judging whether the initial information feature matrix is ​​similar to the preset standard information feature matrix based on a preset matrix similarity judgment method includes: Comparing the initial information feature matrix with the preset standard information feature upper limit matrix and the preset standard information feature lower limit matrix element by element; Determine whether each element in the initial information feature matrix falls within an upper and lower limit interval consisting of a corresponding upper limit and a lower limit; If the above judgment is no, it is determined that the initial information feature matrix is ​​not similar to the preset standard information feature matrix.

9. The method for detecting target in surveillance video based on artificial intelligence according to claim 1, wherein: Judging whether the initial information feature matrix is ​​similar to the preset standard information feature matrix based on a preset matrix similarity judgment method includes: A pre-trained matrix similarity classification model based on deep learning is used to determine whether the initial information feature matrix is ​​similar to the preset standard information feature matrix.

10. A surveillance video target detection system based on artificial intelligence, characterized in that: include: A first acquisition module is configured to acquire an exit monitoring video of the muck truck based on a preset first camera array, wherein the exit monitoring video includes an exit main monitoring video; A first detection module is configured to detect whether the exit main monitoring video contains a muck truck based on a pre-trained muck truck detection model; A first recognition module is configured to, if the above detection result is yes, identify the vehicle identification and vehicle type of the muck truck, and construct an initial information feature matrix of the muck truck based on the exit monitoring video and a preset feature matrix construction method; A first determining module is used to determine a preset standard information feature matrix corresponding to the vehicle type; A first judgment module is configured to judge whether the initial information feature matrix is ​​similar to the preset standard information feature matrix based on a preset matrix similarity judgment method; The first determination module is configured to determine that the target dump truck corresponding to the vehicle identification has no abnormality if the above determination is yes.

Images