Tank car displacement alarm method and system based on image behavior prediction

Abstract

The invention relates to the technical field of charging car safety monitoring, and discloses a tank car displacement alarm method and system based on image behavior prediction, and the method comprises the steps: obtaining the displacement video data of a tank car; processing the displacement video data of the tank car; establishing a tank car interface identification model and a tank car position identification model, analyzing the obtained tank car displacement image data, determining the position of the tank car interface in the tank car displacement image, and if the tank car interface is not identified in the image, identifying the tank car position by using the tank car position identification model; the tank car is controlled to move to a discharging port of the ash silo until the tank car interface in the tank car displacement image is successfully butted; and according to the behavior prediction result, tank car loading is managed. All directions of the tank car are comprehensively covered, and the recognition precision and the butt joint success rate are improved; tank car behaviors are dynamically predicted, and a control strategy is timely adjusted; and the intelligent alarm mechanism is used for intelligent alarm based on the displacement deviation vector, and safe blanking is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring of loading vehicles, and in particular to a method and system for alarming displacement of a tanker based on image behavior prediction. Background Art

[0002] At present, in production practice, the situation of damaging the ash hopper often occurs. After observation and analysis, it is mainly caused by the movement of the tank truck and the operator. There are two reasons: first, when the tank truck moves to the lower ash position, the operator makes a misjudgment and manually lowers the ash hopper; second, due to poor communication, the driver makes a misjudgment and lets the vehicle loaded with ash move.

[0003] In summary, existing monitoring systems may have delays in real-time updating of vehicle location and status data, which is crucial for rapid response to emergencies; monitoring systems on the market may lack comprehensive monitoring capabilities for multiple operating conditions of tank trucks, such as liquid level data, vehicle movement trajectory, driver behavior, etc., which limits the comprehensiveness and intelligence level of the system. Summary of the invention

[0004] The purpose of the present invention is to achieve real-time and accurate monitoring of the entire process of a tank truck through image processing and wireless communication technology, so as to prevent and reduce safety accidents during transportation.

[0005] In order to achieve the above object, the present invention provides a tank truck displacement alarm method based on image behavior prediction, comprising:

[0006] Set up video monitoring points at the ash storage discharge port to obtain tank truck displacement video data;

[0007] By processing the tank truck displacement video data, the tank truck displacement image data is generated;

[0008] Establish tank truck interface recognition model and tank truck position recognition model, and analyze the obtained tank truck displacement image data;

[0009] The tank car interface recognition model recognizes the tank car interface in the tank car displacement image;

[0010] If the tank truck interface is not recognized in the image, the tank truck position recognition model is used to recognize the tank truck position; according to the tank truck position recognition result, the tank truck is controlled to move to the ash bin discharge port until the tank truck interface is recognized in the image;

[0011] Control the movement of the tank truck until the tank truck interface is successfully docked with the ash bin discharge port position, and generate a control discharge instruction.

[0012] In some embodiments of the present invention, when establishing a tank truck interface identification model, the steps include:

[0013] Through simulation experiments, images of the tank truck top under various conditions are generated, and the tank truck interfaces in the images are annotated to obtain the annotated tank truck top images.

[0014] The annotated tank truck top image is preprocessed and features are extracted through convolution operation to obtain the image features of the tank truck interface.

[0015] Based on the image features of the tanker interface, the input image in the tanker interface recognition model is detected to obtain the position P of the tanker interface in the image, where P = [n, x, y, h, w];

[0016] Wherein, the value of n is 1 or 0. When the position of the tanker interface is detected in the image, n=1; when the position of the tanker interface is not detected in the image, n=0.

[0017] x represents the horizontal coordinate of the center point of the identified tank truck interface anchor point box, y represents the vertical coordinate of the center point of the identified tank truck interface anchor point box, h represents the height of the identified tank truck interface anchor point box, and w represents the width of the identified tank truck interface anchor point box.

[0018] In some embodiments of the present invention, when constructing a tank truck position recognition model, the steps include:

[0019] Through simulation tests, historical video data captured by the rear video monitoring point and the video monitoring points on both sides under different conditions during the tank truck docking process are obtained;

[0020] Preprocess the historical video data to obtain the image of the rear of the tank truck and the images of the two sides of the tank truck, and identify the position of the tank truck in the image;

[0021] The coordinate axis is established based on the tank truck docking area, and the coordinates of the tank truck center point and the tank truck boundary information are obtained by combining the image recognition results of the rear of the tank truck and the images on both sides of the tank truck.

[0022] In some embodiments of the present invention, the tank truck is controlled to move to the ash bin discharge port according to the recognition result of the tank truck position until the image recognizes the tank truck interface, including:

[0023] Obtain real-time video data during the tank truck docking process, including the top video data, rear video data, and both side video data of the tank truck;

[0024] Perform real-time analysis on the top video data, identify the position information of the tanker interface in the top video data through the tanker interface recognition model, and obtain the real-time position P' of the tanker interface in the image, P' = [n, x, y, h, w];

[0025] When n = 0, it means that the top video monitoring point does not detect the tanker interface;

[0026] At this time, the rear video data and the video data on both sides are input into the tank truck position recognition model to generate the coordinates of the tank truck center point and the tank truck boundary information, and by comparing with the target position, guide the tank truck to move to the target position;

[0027] When n=1, the position of the tank car interface in the top image is analyzed; based on the values ​​of x, y, h, and w, the contour and position of the tank car interface in the image are determined;

[0028] Compare the outline and position of the tank car interface in the image with the annotated tank car top image to determine whether the position of the tank car interface meets the requirements;

[0029] If it meets the requirements, connect the ash bin discharge port and the tank truck interface;

[0030] If it does not meet the requirements, the video will be sent to the tank truck control end for correction until the tank truck interface is successfully connected.

[0031] In some embodiments of the present invention, the tank truck interface is successfully docked with the ash bin discharge port, including:

[0032] Through the tank car interface recognition model, the position of the tank car interface in the top image is obtained, and the tank car is controlled to move until it reaches the preset position of the tank car interface in the top image, and the docking is successful;

[0033] After successful docking, control unloading instructions are generated.

[0034] In some embodiments of the present invention, controlling the material unloading instruction includes:

[0035] The unloading control end performs unloading operations, and the tanker control end keeps the vehicle stationary;

[0036] During the unloading process, the top video data is monitored in real time, the displacement of the tank truck interface behavior is predicted, and a first-level control strategy is generated based on the results of the displacement prediction to manage the tank truck loading.

[0037] In some embodiments of the present invention, when predicting the displacement of the behavior of the tanker interface, the method includes:

[0038] After successful docking, the tanker interface position A(x,y) is marked in the video data;

[0039] The position of the tank truck interface in the video data is monitored, and based on the obtained real-time position B (x, y) of the tank truck interface and the marked position A (x, y) of the tank truck interface, the displacement offset vector C of the tank truck interface is calculated and generated, C = BA.

[0040] In some embodiments of the present invention, the primary control strategy further includes:

[0041] Set offset thresholds T, including: primary offset threshold T1, secondary offset threshold T2, and tertiary offset threshold T3;

[0042] Perform blanking alarm based on the displacement offset vector and the offset threshold;

[0043] Among them, if |C| < T1, shorten the time interval for video data frame extraction and continue monitoring;

[0044] If T1 < |C| ≤ T2, shorten the time interval for video data frame extraction, and at the same time analyze the change in the vector direction to determine whether to remind the tanker control terminal;

[0045] If T2 < |C| ≤ T3, send a message to the tanker control terminal for reminder;

[0046] If T3 < |C|, stop the blanking process.

[0047] In some embodiments of the present invention, if T1 < |C| ≤ T2, it further includes:

[0048] Taking the center point of the tanker position as the origin, establish a plane rectangular coordinate system, and obtain n offset vectors C, C = [C1, C2…C i …C n ;

[0049] Among them, C1 represents the first obtained offset vector, C i represents the i-th obtained offset vector, i ≠ 1, and n represents the number of obtained offset vectors;

[0050] Calculate the angle difference p between C i and C1 for offset judgment;

[0051] If p < 90, it is determined that the vehicle is offset, and a reminder message is sent to the tanker control terminal;

[0052] If p ≥ 90, it is determined that the vehicle is not offset, and n offset vectors C are obtained again for angle difference p calculation and offset judgment.

[0053] A tanker displacement alarm system based on image behavior prediction, applying the above-mentioned tanker displacement alarm method based on image behavior prediction, includes:

[0054] A data acquisition module for acquiring tanker displacement video data;

[0055] A data preprocessing module for processing the tanker displacement video data to obtain tanker displacement image data;

[0056] A data analysis module for establishing a tanker interface recognition model and a tanker position recognition model to determine the position of the tanker and the tanker interface;

[0057] The behavior prediction module is used to analyze the position of the tank truck interface in the image to determine whether it corresponds to the ash bin discharge port position, and to control the discharge after successful connection;

[0058] The behavior of the interface position is predicted, and the tank truck loading is managed based on the behavior prediction results.

[0059] Compared with the prior art, the method and system for tank truck displacement alarm based on image behavior prediction provided by the embodiment of the present invention have the following beneficial effects:

[0060] The present invention uses image recognition and behavior analysis algorithms to automatically detect the state and movement trajectory of the tank truck and warn of abnormal situations in advance.

[0061] The present invention establishes an effective feedback mechanism to ensure rapid adjustment and precise alignment of the tank truck position and stable operation of the docking system under various conditions, thereby avoiding docking failures caused by system failures.

[0062] In the present invention, remote access and monitoring are realized by setting monitoring points, so that the status of the tank truck can be checked in real time and emergency events can be responded to quickly.

[0063] The present invention effectively prevents safety accidents caused by displacement of the tank truck interface through real-time monitoring and early warning as well as intelligent monitoring strategies; avoids unnecessary stopping and starting, and improves loading efficiency.

[0064] The present invention effectively distinguishes between deviation and normal jitter through accurate identification of direction changes to avoid false alarms; and ensures that the tank truck control end can receive vehicle deviation information in a timely manner and make adjustments quickly through an intelligent monitoring and response mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flow chart of a tank truck displacement alarm method based on image behavior prediction provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0067] In the description of the present invention, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0068] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0069] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0070] Embodiment 1:

[0071] The preferred embodiment of the present invention is a tank truck displacement alarm method based on image behavior prediction, such as Figure 1 As shown, including:

[0072] Set up video monitoring points at the ash storage discharge port to obtain tank truck displacement video data;

[0073] By processing the tank truck displacement video data, the tank truck displacement image data is generated;

[0074] Establish tank truck interface recognition model and tank truck position recognition model, and analyze the obtained tank truck displacement image data;

[0075] Determine the tank truck interface in the tank truck displacement image. If the tank truck interface is not identified in the image, use the tank truck position recognition model to identify the tank truck position. According to the tank truck position recognition result, control the tank truck to move to the ash bin discharge port until the image recognizes the tank truck interface.

[0076] By detecting the position of the tank car interface in the tank car displacement video data, it is determined whether the tank car interface is successfully connected with the ash bin discharge port position, and a control discharge instruction is generated after successful connection;

[0077] By predicting the behavior of the tank truck interface, a tank truck interface behavior prediction result is generated, and a first-level control instruction is generated according to the tank truck interface behavior prediction result.

[0078] Embodiment 2, a high-precision video monitoring point group is arranged at the ash storage discharge port to obtain the tank truck displacement video data, including:

[0079] Taking the position of the tanker when the tanker interface is connected to the ash bin discharge port as the center point, the video monitoring points of the ash bin discharge port are:

[0080] An upper video monitoring point group arranged just above the tanker interface;

[0081] A group of video monitoring points arranged on both sides of the tanker;

[0082] A rear video monitoring point group arranged behind the tank truck;

[0083] Among them, the upper video monitoring point group is used to obtain the video data of the top of the tank truck in the image, and the rear video monitoring point group and the two side video monitoring point groups are used to obtain the position video data of the tank truck in the working area.

[0084] In this embodiment, the upper video monitoring point group is installed directly above the tank truck interface to ensure that the field of view covers the tank mouth and the surrounding area, mainly capturing the top features of the tank truck to facilitate interface identification.

[0085] Video monitoring point groups on both sides: deployed on both sides of the tank truck, covering the side of the tank body, monitoring the lateral movement and position of the tank truck, and used for positioning and behavior analysis of the tank truck.

[0086] Rear video monitoring point group: located behind the tank truck, capturing the dynamics of the rear of the tank truck, assisting in the longitudinal positioning and motion trajectory analysis of the tank truck.

[0087] In this embodiment, multi-angle coverage: ensure that all video monitoring points are started synchronously, realize all-round and no-dead-angle video acquisition, and provide full support for subsequent data analysis.

[0088] Timestamp synchronization: All video monitoring points should be set with a unified timestamp to ensure the temporal consistency of the data and facilitate subsequent behavior analysis and event tracing.

[0089] Video quality control: High resolution and frame rate settings to ensure image clarity and dynamic capture capabilities and reduce motion blur.

[0090] In this embodiment, regular calibration: video monitoring points need to be calibrated regularly to ensure the accuracy of position and angle and avoid image deviation.

[0091] System maintenance: Establish a regular maintenance plan to check network connections and equipment operating status to ensure stable system operation.

[0092] Embodiment 3, when establishing a tank truck interface identification model, includes:

[0093] Through simulation experiments, images of the tank truck top under various conditions are generated, and the tank truck interfaces in the images are annotated to obtain the annotated tank truck top images.

[0094] The annotated tank truck top image is preprocessed and features are extracted through convolution operation to obtain the image features of the tank truck interface.

[0095] Based on the image features of the tanker interface, the input image in the tanker interface recognition model is detected to obtain the position P of the tanker interface in the image, where P = [n, x, y, h, w];

[0096] Wherein, the value of n is 1 or 0. When the position of the tanker interface is detected in the image, n=1; when the position of the tanker interface is not detected in the image, n=0.

[0097] x represents the horizontal coordinate of the center point of the identified tank truck interface anchor point box, y represents the vertical coordinate of the center point of the identified tank truck interface anchor point box, h represents the height of the identified tank truck interface anchor point box, and w represents the width of the identified tank truck interface anchor point box.

[0098] In this embodiment, a test platform for simulating the docking process between a tank truck and an ash bin discharge port is designed.

[0099] Tests were conducted under different lighting conditions and tanker states (empty, half-loaded, fully loaded) to ensure data diversity.

[0100] Use the upper video monitoring point group to capture images of the tank truck top, covering different angles and postures of the interface.

[0101] Manually mark the tanker interface locations, including the center point (x, y) and bounding box size (h, w).

[0102] Use a pre-trained model of a convolutional neural network (such as ResNet, VGG) and use its convolutional layer to extract image features.

[0103] The preprocessed image is input into the model to extract the image features of the tank truck interface.

[0104] Use an annotated dataset to train the model and adjust the weights to minimize the loss function, such as cross entropy loss. Use stochastic gradient descent (SGD) or Adam optimizer and set appropriate learning rate, momentum and other parameters. Divide the dataset into training, validation and test sets, monitor the performance of the model on the validation set and prevent overfitting.

[0105] Embodiment 4, when building a tank truck position recognition model, includes:

[0106] Through simulation tests, historical video data captured by the rear video monitoring point and the video monitoring points on both sides under different conditions during the tank truck docking process are obtained;

[0107] Preprocess the historical video data to obtain the image of the rear of the tank truck and the images of the two sides of the tank truck, and identify the position of the tank truck in the image;

[0108] The coordinate axis is established based on the tank truck docking area, and the coordinates of the tank truck center point and the tank truck boundary information are obtained by combining the image recognition results of the rear of the tank truck and the images on both sides of the tank truck.

[0109] In this embodiment, the simulation test:

[0110] Create a simulation environment for the tank truck docking process, covering a variety of lighting conditions, different time points, and different states of the tank truck (such as empty, half-loaded, and fully loaded); use the rear video monitoring points and the video monitoring point groups on both sides to capture video data of the tank truck during the docking process from multiple angles; collect historical video data of the tank truck docking process from simulation tests; and ensure that the data set covers the movement state of the tank truck under different conditions.

[0111] Using Yolov5 object detection:

[0112] The Yolov5 model is used to perform target detection on the processed images and identify the position of the tank truck; the Yolov5 model has high accuracy in target detection in industrial scenes and is particularly suitable for object recognition in complex backgrounds.

[0113] Coordinate axis establishment:

[0114] A coordinate axis is established in the tanker docking area, with the tanker docking point as the origin and the x and y axes defined; this helps to accurately describe the position and motion trajectory of the tanker in the docking area.

[0115] Location and boundary information extraction:

[0116] The detection results of the Yolov5 model are used to extract the coordinates of the center point and boundary information of the tank truck (including the length and width of the tank truck); the precise position of the tank truck in the docking area is obtained through comprehensive analysis of the recognition results of the rear image and the images on both sides of the tank truck.

[0117] Embodiment 5, controlling the tank truck to move to the ash bin discharge port according to the recognition result of the tank truck position, until the image recognizes the tank truck interface, includes:

[0118] Obtain real-time video data during the tank truck docking process, including the top video data, rear video data, and both side video data of the tank truck;

[0119] Perform real-time analysis on the top video data, identify the position information of the tanker interface in the top video data through the tanker interface recognition model, and obtain the real-time position P' of the tanker interface in the image, P' = [n, x, y, h, w];

[0120] When n = 0, it means that the top video monitoring point does not detect the tanker interface;

[0121] At this time, the rear video data and the video data on both sides are input into the tank truck position recognition model to generate the coordinates of the tank truck center point and the tank truck boundary information, and by comparing with the target position, guide the tank truck to move to the target position;

[0122] When n=1, the position of the tanker interface in the top image is analyzed; based on the values ​​of x, y, h, and w, the contour and position of the tanker interface in the image are determined;

[0123] Compare the outline and position of the tank car interface in the image with the annotated tank car top image to determine whether the position of the tank car interface meets the requirements;

[0124] If it meets the requirements, connect the ash bin discharge port and the tank truck interface;

[0125] If it does not meet the requirements, the video will be sent to the tank truck control end for correction until the tank truck interface is successfully connected.

[0126] In this embodiment, the real-time video analysis and position adjustment are ensured, and the position changes of the tank truck are responded to quickly. The tank truck interface identification and position information analysis need to be highly accurate to ensure successful docking. A fast and effective feedback mechanism is established between the tank truck and the control system to ensure the timely transmission of adjustment instructions. The system needs to operate stably under various conditions, including different lighting and weather conditions.

[0127] Example 6, the tank truck interface is successfully connected to the ash bin discharge port, including:

[0128] Through the tank car interface recognition model, the position of the tank car interface in the top image is obtained, and the tank car is controlled to move until it reaches the preset position of the tank car interface in the top image, and the docking is successful;

[0129] After successful docking, control unloading instructions are generated.

[0130] Control unloading instructions, including:

[0131] The unloading control end performs unloading operations, and the tanker control end keeps the vehicle stationary;

[0132] During the unloading process, the top video data is monitored in real time, the displacement of the tank truck interface behavior is predicted, and a first-level control strategy is generated based on the results of the displacement prediction to manage the tank truck loading.

[0133] Embodiment 7, performing displacement prediction on the behavior of the tank truck interface, comprises:

[0134] After successful docking, the tanker interface position A(x,y) is marked in the video data;

[0135] Monitor the position of the tanker interface in the video data. Based on the obtained real-time position B(x, y) of the tanker interface and the marked position A(x, y) of the tanker interface, calculate the displacement offset vector C of the tanker interface, where C = B - A;

[0136] Example 8, primary control strategy. Set offset thresholds T, including: primary offset threshold T1, secondary offset threshold T2, and tertiary offset threshold T3;

[0137] Conduct blanking alarm based on the displacement offset vector and the offset threshold;

[0138] Among them, if |C| < T1, shorten the time interval of video data frame extraction and maintain monitoring;

[0139] If T1 < |C| ≤ T2, shorten the time interval of video data frame extraction and simultaneously analyze the change in the vector direction to determine whether to remind the tanker control terminal;

[0140] If T2 < |C| ≤ T3, send a message to the tanker control terminal for reminder;

[0141] If T3 < |C|, stop the blanking process.

[0142] Example 9, conduct blanking alarm based on the displacement offset vector and the offset threshold;

[0143] Among them, if |C| < T1, shorten the time interval of video data frame extraction and maintain monitoring;

[0144] If T1 < |C| ≤ T2, shorten the time interval of video data frame extraction and simultaneously analyze the change in the vector direction to determine whether to remind the tanker control terminal;

[0145] If T2 < |C| ≤ T3, send a message to the tanker control terminal for reminder;

[0146] If T3 < |C|, stop the blanking process.

[0147] In this example, for video data processing: adopt a high-speed video stream to ensure real-time performance, and at the same time, through algorithm optimization, reduce video processing delay; for displacement calculation: use image processing technology to accurately calculate the real-time position of the tanker interface to ensure the accuracy of displacement calculation; for threshold dynamic adjustment: dynamically adjust the offset threshold according to the tanker type and actual requirements during the loading process to improve the flexibility and effectiveness of monitoring.

[0148] Implementation cases

[0149] Primary alarm case: At the initial stage of blanking, the tanker interface moves slightly. The system automatically shortens the frame extraction time and continuously monitors without triggering a higher-level alarm.

[0150] Level 3 alarm case: The tank truck is affected by external factors during unloading and the interface displacement exceeds T2. The system immediately reminds the tank truck control end and the operator makes timely adjustments to avoid accidents.

[0151] Improved safety: Real-time monitoring and early warning can effectively prevent safety accidents caused by displacement of tanker interfaces; Efficiency optimization: Intelligent monitoring strategies can avoid unnecessary stops and starts, and improve loading efficiency; Cost reduction: Reduce equipment damage and material waste caused by accidents, and significantly reduce costs in long-term operation.

[0152] Embodiment 10, if T1<|C|≤T2, further comprising:

[0153] Taking the center point of the tanker as the origin, establish a plane rectangular coordinate system and obtain the n-th offset vector C, C = [C1, C2…C i …C n ];

[0154] Among them, C1 represents the offset vector obtained for the first time, C i represents the offset vector obtained for the i-th time, i≠1, and n represents the number of times the offset vector is obtained;

[0155] Calculate C i The angle difference p with C1 is used to determine the offset;

[0156] If p<90, the vehicle is judged to be deviating and a warning message is sent to the tanker control terminal;

[0157] If p ≥ 90, it is determined that the vehicle has not deviated, and the offset vector C is obtained n times again, the angle difference p is calculated, and the offset judgment is performed.

[0158] In this embodiment, the continuously measured offset vectors are used to calculate the angle change between adjacent vectors, and the change angle is determined by trigonometric calculation or vector dot product. The system sets the angle threshold to 90° as a criterion for distinguishing between offset and jitter.

[0159] Case 1 (offset): During a loading process, the multiple measurements of the displacement offset vector of the tanker interface showed stable changes in direction, and the angle was always less than 90°. The system determined that the vehicle was offset to one side and immediately sent a warning message to the tanker control end. The operator adjusted the tanker position in time to avoid potential risks.

[0160] Case 2 (jitter): During another loading process, the measurement direction of the displacement offset vector of the tanker interface changed frequently, and the angle exceeded 90° many times. The system determined it to be normal jitter, did not send a reminder message, and maintained monitoring to ensure the normal progress of the loading process.

[0161] Accurate monitoring: Through accurate identification of directional changes, it can effectively distinguish between deviation and normal jitter to avoid false alarms; instant response: real-time monitoring and instant response mechanism ensure that the tank truck control end can receive vehicle deviation information in time and make rapid adjustments; safety improvement: the implementation of intelligent judgment and control strategies significantly improves the safety of the loading process.

[0162] A data acquisition module, used to acquire the tank truck displacement video data;

[0163] A data preprocessing module is used to process the tank truck displacement video data to obtain the tank truck displacement image data;

[0164] A data analysis module is used to establish a tank truck interface identification model and a tank truck position identification model to determine the positions of the tank truck and the tank truck interface;

[0165] The behavior prediction module is used to analyze the position of the tank truck interface in the image to determine whether it corresponds to the ash bin discharge port position, and to control the discharge after successful connection;

[0166] The behavior of the interface position is predicted, and the tank truck loading is managed based on the behavior prediction results.

[0167] Finally, it should be noted that it is obvious that those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalent technology, the present invention is also intended to include these modifications and variations.

[0168] The above is only an example of implementation of the present invention, but it cannot be used to limit the scope of the present invention. Any structural changes made according to the present invention, as long as they do not lose the essence of the present invention, should be regarded as falling within the scope of protection of the present invention and being restricted. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and related instructions of the platform described above can refer to the corresponding process in the aforementioned platform embodiment, and will not be repeated here.

[0169] The term "comprising" or any other similar term is intended to cover a non-exclusive inclusion such that a process, platform, article, or apparatus / platform that comprises a list of elements includes not only those elements but also other elements not expressly listed or also includes elements inherent to such process, platform, article, or apparatus / platform.

[0170] So far, the technical solutions of the present invention have been described in conjunction with the further embodiments shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0171] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A tank truck displacement alarm method based on image behavior prediction, characterized in that: include: Set up video monitoring points at the ash storage discharge port to obtain tank truck displacement video data; By processing the tank truck displacement video data, the tank truck displacement image data is generated; Establish tank truck interface recognition model and tank truck position recognition model, and analyze the obtained tank truck displacement image data; The tank car interface recognition model recognizes the tank car interface in the tank car displacement image; If the tank truck interface is not recognized in the image, the tank truck position recognition model is used to recognize the tank truck position; according to the tank truck position recognition result, the tank truck is controlled to move to the ash bin discharge port until the tank truck interface is recognized in the image; Control the movement of the tank truck until the tank truck interface is successfully docked with the ash bin discharge port position, and generate a control discharge instruction.

2. The tank truck displacement alarm method based on image behavior prediction according to claim 1 is characterized in that: When establishing a tank truck interface identification model, it includes: Through simulation experiments, images of the tank truck top under various conditions are generated, and the tank truck interfaces in the images are annotated to obtain the annotated tank truck top images. The annotated tank truck top image is preprocessed and features are extracted through convolution operation to obtain the image features of the tank truck interface. Based on the image features of the tanker interface, the input image in the tanker interface recognition model is detected to obtain the position P of the tanker interface in the image, where P = [n, x, y, h, w]; Wherein, the value of n is 1 or 0. When the position of the tanker interface is detected in the image, n=1; when the position of the tanker interface is not detected in the image, n=0. x represents the horizontal coordinate of the center point of the identified tank truck interface anchor point box, y represents the vertical coordinate of the center point of the identified tank truck interface anchor point box, h represents the height of the identified tank truck interface anchor point box, and w represents the width of the identified tank truck interface anchor point box.

3. The tank truck displacement alarm method based on image behavior prediction according to claim 2 is characterized in that: When building a tank truck location recognition model, it includes: Through simulation tests, historical video data captured by the rear video monitoring point and the video monitoring points on both sides under different conditions during the tank truck docking process are obtained; Preprocess the historical video data to obtain the image of the rear of the tank truck and the images of the two sides of the tank truck, and identify the position of the tank truck in the image; The coordinate axis is established based on the tank truck docking area, and the coordinates of the tank truck center point and the tank truck boundary information are obtained by combining the image recognition results of the rear of the tank truck and the images on both sides of the tank truck.

4. The tank truck displacement alarm method based on image behavior prediction according to claim 3 is characterized in that: According to the identification result of the tanker position, the tanker is controlled to move to the ash bin discharge port until the image recognizes the tanker interface, including: Obtain real-time video data during the tank truck docking process, including the top video data, rear video data, and both side video data of the tank truck; Perform real-time analysis on the top video data, identify the position information of the tanker interface in the top video data through the tanker interface recognition model, and obtain the real-time position P' of the tanker interface in the image, P' = [n, x, y, h, w]; When n = 0, it means that the top video monitoring point does not detect the tanker interface; At this time, the rear video data and the video data on both sides are input into the tank truck position recognition model to generate the coordinates of the tank truck center point and the tank truck boundary information, and by comparing with the target position, guide the tank truck to move to the target position; When n=1, the position of the tanker interface in the top image is analyzed; based on the values ​​of x, y, h, and w, the contour and position of the tanker interface in the image are determined; Compare the outline and position of the tanker interface in the image with the marked image of the tanker top to determine whether the position of the tanker interface meets the requirements; If it meets the requirements, connect the ash silo discharge port to the tanker interface; If it does not meet the requirements, send the video to the tanker control terminal for correction until the tanker interface is successfully docked.

5. The method for tanker displacement alarm based on image behavior prediction according to claim 4 is characterized in that: The successful docking of the tanker interface and the ash silo discharge port includes: Through the tanker interface recognition model, obtain the position of the tanker interface in the top image, control the movement of the tanker until the position of the tanker interface in the top image reaches the preset position, and the docking is successful; After the docking is successful, generate a control feeding instruction.

6. The method for tanker displacement alarm based on image behavior prediction according to claim 5 is characterized in that: The control feeding instruction includes: The feeding control terminal performs the feeding operation, and the tanker control terminal keeps the vehicle stationary; During the feeding process, monitor the top video data in real time, perform displacement prediction on the behavior of the tanker interface, generate a primary control strategy based on the result of the displacement prediction, and manage the tanker loading.

7. The method for tanker displacement alarm based on image behavior prediction according to claim 6, characterized in that: When performing displacement prediction on the behavior of the tanker interface, it includes: After the docking is successful, mark the position A(x,y) of the tanker interface in the video data; Monitor the position of the tanker interface in the video data, and calculate and generate the displacement offset vector C of the tanker interface based on the obtained real-time position B(x,y) of the tanker interface and the marked position A(x,y) of the tanker interface, C = B - A.

8. The method for tanker displacement alarm based on image behavior prediction according to claim 7 is characterized in that: The primary control strategy also includes: Set the offset threshold T, including: the primary offset threshold T1, the secondary offset threshold T2, and the tertiary offset threshold T3; Perform feeding alarm based on the displacement offset vector and the offset threshold; Among them, if |C| < T1, shorten the time interval of video data frame extraction and keep monitoring; If T1 < |C| ≤ T2, shorten the time interval of video data frame extraction, and at the same time analyze the change of the vector direction to judge whether it is necessary to remind the tanker control terminal; If T2 < |C| ≤ T3, send a message to the tanker control terminal for reminder; T3 < |C|, stop the feeding process.

9. The method for tanker displacement alarm based on image behavior prediction according to claim 8, characterized in that: If T1 < |C| ≤ T2, it also includes: Taking the center point of the tanker as the origin, establish a plane rectangular coordinate system and obtain the n-time offset vector C, C = [C1, C2…C i …C n ]; Among them, C1 represents the offset vector obtained for the first time, C i represents the offset vector obtained for the i-th time, i≠1, and n represents the number of times the offset vector is obtained; Calculate C i The angle difference p with C1 is used to determine the offset; If p < 90, judge that the vehicle is offset and send a reminder message to the tanker control terminal; If p ≥ 90, judge that the vehicle is not offset, obtain the offset vector C n times again, calculate the angle difference p, and perform offset judgment.

10. A tank truck displacement alarm system based on image behavior prediction, characterized in that: Applying the tanker displacement alarm method based on image behavior prediction described in any one of the above claims 1-9, includes: A data acquisition module for acquiring tanker displacement video data; A data preprocessing module for processing the tanker displacement video data to obtain tanker displacement image data; A data analysis module for establishing a tanker interface recognition model and a tanker position recognition model to determine the position of the tanker and the tanker interface; A behavior prediction module for analyzing the position of the tanker interface in the image, judging whether it corresponds to the position of the ash silo discharge port, and performing feeding control after successful docking; Perform behavior prediction on the interface position, and manage the tanker loading according to the result of the behavior prediction.

Images