A transportation monitoring method, system and electronic equipment for a mixer truck

Abstract

The present application provides a method, system and electronic equipment for monitoring the transportation of a mixer truck, which relates to the technical field of operating machinery, and includes constructing a task to be executed, wherein the task to be executed includes: a target mixer truck and a target position; obtaining the tank video information of the target mixer truck; the tank video information includes: a tank side image and a tank tail image; detecting the tank side image through a rotation detection model to determine whether the rotation direction of the tank of the target mixer truck has changed; when the rotation direction of the tank has changed, identifying the slump feature from the tank tail image through a concrete recognition model, and determining whether the concrete in the target mixer truck has been poured out according to the slump feature; when it is detected that the concrete in the target mixer truck has been poured out, determining whether the target mixer truck is unloading normally based on the consistency between the actual position of the target mixer truck and the target position, and issuing an early warning based on the determination result, so as to realize real-time monitoring of the transportation process of the mixer truck.

Description

Technical Field

[0001] The present invention relates to the technical field of operating machinery, and in particular to a transportation monitoring method, system, electronic equipment and computer-readable storage medium for a mixer truck. Background Art

[0002] After the concrete is produced at the mixing plant, it needs to be transported to the construction site by a mixer truck (i.e., a concrete mixer truck) for pumping, pouring, etc. However, during the transportation of concrete, abnormal unloading may occur, which will not only cause economic losses, but also affect the quality of the project.

[0003] Therefore, it is necessary to provide a transportation monitoring method, system, electronic device and computer-readable storage medium for a mixer truck. Summary of the invention

[0004] The present specification provides a method, system, electronic device and computer-readable storage medium for monitoring the transportation of a mixer truck. By acquiring the tank video information of the target mixer truck and analyzing the rotation direction of the tank and the slump characteristics of the concrete in real time, it is possible to promptly detect whether the concrete has been poured out, thereby achieving real-time monitoring of the mixer truck transportation process, helping to promptly detect potential illegal unloading behaviors or unexpected situations and improving transportation safety.

[0005] The present application provides a method for monitoring the transportation of a mixer truck, which adopts the following technical solution, including:

[0006] Constructing a task to be performed, wherein the task to be performed includes: a target mixer truck and a target position;

[0007] Obtaining the tank video information of the target mixer truck; the tank video information includes: a tank side image and a tank tail image;

[0008] The tank side image is detected by a rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck has changed;

[0009] When the rotation direction of the tank body changes, the slump feature is identified from the tank tail image by a concrete recognition model, and whether the concrete in the target mixer truck has been poured out is determined according to the slump feature;

[0010] When it is detected that the concrete in the target mixer truck is poured out, it is judged whether the target mixer truck is unloading normally based on the consistency between the actual position of the target mixer truck and the target position, and an early warning is issued according to the judgment result.

[0011] Optionally, detecting the tank side image by a rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck has changed includes:

[0012] Acquire a current tank side frame image of the tank side image;

[0013] Performing key point recognition on the current tank side frame image to obtain current features of the key points;

[0014] Combining the current feature of the key point with the previous feature of the key point, it is determined whether the rotation direction of the tank body has changed.

[0015] Optionally, the step of performing key point recognition on the current tank side frame image to obtain current features of the key points includes:

[0016] Performing a first preprocessing operation on the current tank side frame image to obtain the current tank side frame image after edge detection;

[0017] Performing graphic detection on the current tank side frame image after the edge detection to extract key lines of the tank body;

[0018] Combined with the key lines of the tank body, the current features of the key points are constructed.

[0019] Optionally, performing a first preprocessing operation on the current tank side frame image to obtain the current tank side frame image after edge detection includes:

[0020] Performing a first grayscale processing on the current tank side frame image to obtain a grayscaled current tank side frame image;

[0021] Smoothing the grayscaled current tank side frame image to obtain a smoothed current tank side frame image;

[0022] The smoothed current tank side frame image is subjected to edge detection to determine the overall contour of the tank body, thereby generating the current tank side frame image after edge detection.

[0023] Optionally, when the rotation direction of the tank body changes, identifying a slump feature from the tank tail image by using a concrete recognition model, and judging whether the concrete in the target mixer truck has been poured out according to the slump feature, includes:

[0024] Obtaining a current can tail frame image of the can tail image;

[0025] Performing concrete recognition on the current tank tail frame image to extract slump characteristics of the target area;

[0026] In combination with the slump characteristics and the preset slump conditions, it is determined whether the concrete has been poured out.

[0027] Optionally, the step of performing concrete recognition on the current tank tail frame image to extract slump characteristics of a target area includes:

[0028] Performing a second preprocessing operation on the current can tail frame image to obtain a preprocessed current can tail frame image;

[0029] Feature extraction is performed on the preprocessed current tank tail frame image to obtain the slump feature of the target area.

[0030] Optionally, the performing feature extraction on the preprocessed current tank tail frame image to obtain the slump feature of the target area includes:

[0031] In combination with the pre-processed current tank tail frame image, analyzing whether the target area has dynamic changes;

[0032] When dynamic changes occur, detecting whether there is a target object under the tank;

[0033] When the target object appears, the flow morphology of the target object is identified and the slump feature is generated.

[0034] The present application provides a transport monitoring system for a mixer truck, which adopts the following technical solutions, including:

[0035] A task construction module is used to construct a task to be executed, wherein the task to be executed includes: a target mixer truck and a target location;

[0036] A video acquisition module is used to acquire the tank video information of the target mixer truck; the tank video information includes: a tank side image and a tank tail image;

[0037] A rotation detection module, used to detect the tank side image through a rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck has changed;

[0038] A slump detection module is used to identify the slump characteristics from the image of the tank tail through a concrete recognition model when the rotation direction of the tank body changes, and determine whether the concrete in the target mixer truck has been poured out according to the slump characteristics;

[0039] The position detection module is used to determine whether the target mixer truck is unloading normally based on the consistency between the actual position of the target mixer truck and the target position when it is detected that the concrete in the target mixer truck is poured out, and to issue an early warning according to the judgment result.

[0040] Optionally, the rotation detection module includes:

[0041] A tank side frame image acquisition submodule, used for acquiring a current tank side frame image of the tank side image;

[0042] A key point recognition submodule is used to perform key point recognition on the current tank side frame image to obtain current features of the key points;

[0043] The rotation judgment submodule is used to judge whether the rotation direction of the tank body has changed by combining the current feature of the key point and the previous feature of the key point.

[0044] Optionally, the key point identification submodule includes:

[0045] A first preprocessing unit, configured to perform a first preprocessing operation on the current tank side frame image to obtain the current tank side frame image after edge detection;

[0046] A graphics detection unit, used to perform graphics detection on the current tank side frame image after the edge detection, and extract key lines of the tank body;

[0047] The feature construction unit is used to construct the current features of the key points in combination with the key lines of the tank body.

[0048] Optionally, the first preprocessing unit includes:

[0049] A first grayscale subunit, configured to perform a first grayscale process on the current tank side frame image to obtain a grayscaled current tank side frame image;

[0050] A smoothing processing subunit, used for performing smoothing processing on the grayscale current tank side frame image to obtain a smoothed current tank side frame image;

[0051] The edge detection subunit is used to perform edge detection on the smoothed current tank side frame image, determine the overall contour of the tank body, and generate the current tank side frame image after edge detection.

[0052] Optionally, the slump detection module includes:

[0053] A can tail frame image acquisition submodule is used to acquire a current can tail frame image of the can tail image;

[0054] A concrete recognition submodule, used to perform concrete recognition on the current tank tail frame image and extract the slump characteristics of the target area;

[0055] The preset feature judgment submodule is used to judge whether the concrete has been poured out by combining the slump feature and the preset slump condition.

[0056] Optionally, the concrete identification submodule includes:

[0057] A second preprocessing unit, configured to perform a second preprocessing operation on the current can tail frame image to obtain a preprocessed current can tail frame image;

[0058] The slump determination unit is used to extract features from the preprocessed current can tail frame image to obtain the slump features of the target area.

[0059] Optionally, the slump determination unit comprises:

[0060] A dynamic change analysis subunit, used to analyze whether there is a dynamic change in the target area in combination with the pre-processed current tank tail frame image;

[0061] A target object detection subunit, used for detecting whether there is a target object under the tank body when dynamic changes occur;

[0062] The slump feature generating subunit is used to identify the flow form of the target object and generate the slump feature when the target object appears.

[0063] This specification also provides an electronic device, wherein the electronic device includes:

[0064] processor; and,

[0065] A memory storing computer executable instructions, which when executed cause the processor to perform any of the above methods.

[0066] The present specification also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, any of the above methods is implemented.

[0067] In the present application, by constructing a task to be executed, the task to be executed includes: a target mixer truck and a target position; obtaining the tank video information of the target mixer truck; the tank video information includes: a tank side image and a tank tail image; detecting the tank side image through a rotation detection model to determine whether the rotation direction of the tank of the target mixer truck has changed; when the rotation direction of the tank has changed, identifying the slump feature from the tank tail image through a concrete recognition model, and determining whether the concrete in the target mixer truck has been poured out according to the slump feature; when it is detected that the concrete in the target mixer truck has been poured out, determining whether the target mixer truck is unloading normally based on the consistency between the actual position of the target mixer truck and the target position, and issuing an early warning based on the determination result; by determining the unloading status of the mixer truck, the need for manual monitoring is reduced, and management costs are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic diagram of the principle of a method for transport monitoring of a mixer truck provided in an embodiment of this specification;

[0069] Figure 2 A schematic diagram of a left camera viewing angle of a method for transport monitoring of a mixer truck provided in an embodiment of this specification;

[0070] Figure 3 A schematic diagram of a right camera viewing angle of a method for transport monitoring of a mixer truck provided in an embodiment of this specification;

[0071] Figure 4 A schematic diagram of the rear camera viewing angle of a method for transport monitoring of a mixer truck provided in an embodiment of this specification;

[0072] Figure 5 A schematic diagram of the flow chart of S3 of a method for monitoring the transportation of a mixer truck provided in an embodiment of this specification;

[0073] Figure 6 A schematic diagram of the flow chart of S4 of a method for monitoring the transportation of a mixer truck provided in an embodiment of this specification;

[0074] Figure 7 A schematic diagram of the structure of a transport monitoring system for a mixer truck provided in an embodiment of this specification;

[0075] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0076] Fig. 9 A schematic diagram of a computer-readable medium provided for an embodiment of this specification. DETAILED DESCRIPTION

[0077] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0078] Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in a variety of forms, and should not be construed as limiting the present invention to the embodiments set forth herein. On the contrary, providing these exemplary embodiments enables the present invention to be more comprehensive and complete, and is more convenient for fully conveying the inventive concept to those skilled in the art. The same reference numerals in the figures represent the same or similar elements, components or parts, and thus their repeated description will be omitted.

[0079] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0080] In the description of specific embodiments, the features, structures, characteristics or other details described in the present invention are intended to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can practice the technical solutions of the present invention without one or more of the specific features, structures, characteristics or other details.

[0081] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0082] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0083] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0084] Figure 1 A schematic diagram of the principle of a method for monitoring the transportation of a mixer truck provided in an embodiment of this specification, the method comprising:

[0085] S1 constructs a task to be executed, wherein the task to be executed includes: a target mixer truck and a target position;

[0086] S2 obtains the tank video information of the target mixer truck; the tank video information includes: a tank side image and a tank tail image;

[0087] S3 detects the tank side image through a rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck has changed;

[0088] S4: when the rotation direction of the tank body changes, a slump feature is identified from the tank tail image by using a concrete recognition model, and whether the concrete in the target mixer truck has been poured out is determined according to the slump feature;

[0089] S5: When it is detected that the concrete in the target mixer truck is poured out, based on the consistency between the actual position of the target mixer truck and the target position, it is determined whether the target mixer truck is unloading normally, and an early warning is issued according to the determination result.

[0090] A mixer truck is a special truck used to transport ready-mixed concrete for construction. It is equipped with a mixing tank, and the tank body of the mixing tank is used to load the mixed concrete. During transportation, the tank body will continue to rotate to ensure that the concrete remains in a fluid state and prevent it from solidifying and affecting the use effect. The head of the tank body is close to the driver's cabin of the mixer truck, and the tail of the tank body is far away from the driver's cabin of the mixer truck. The entire tank body can rotate around the axis from the head to the tail.

[0091] During the entire transportation process of concrete, abnormal unloading may occur. Abnormal unloading refers to unloading at a location other than the designated location, which may involve: transporting concrete to other locations for feeding; dumping concrete residues at will, etc.

[0092] If concrete is transported to other locations for feeding, the original project will be short of materials, thus affecting the quality of the project. If the leftover concrete is dumped at will, the company may be punished and suffer economic losses. Therefore, in order to ensure the standardization and safety of concrete transportation, the present invention provides a transportation monitoring method for a mixer truck. Specifically, the method includes:

[0093] S1 constructs a task to be executed, wherein the task to be executed includes: a target mixer truck and a target position;

[0094] Specifically, S11 determines the total concrete production volume and target location based on the contract and related projects, and creates a task order.

[0095] After the task order is created, concrete production starts according to the task order, wherein the target location is the unloading area of ​​the target construction site.

[0096] S12 performs scheduling based on the task list and the current transport capacity of the mixer truck to determine each target mixer truck; and sends the corresponding task to be executed to each target mixer truck.

[0097] There is a one-to-one correspondence between a mixer truck and a driver. During scheduling, tasks are assigned to specific target mixer trucks. After tasks to be performed are generated, the driver corresponding to the target mixer truck is informed of the basic information of the tasks to be performed, and the driver is notified to drive the target mixer truck to queue up at the mixing station to load concrete.

[0098] Preferably, a task list of tasks to be performed is pushed to the driver corresponding to the target mixer truck; the task list of tasks to be performed displays basic information of the tasks to be performed;

[0099] In one embodiment of the present specification, the basic information of the task to be executed includes: the target construction site address and the loading amount of the target mixer truck.

[0100] In another embodiment of the present specification, the basic information of the task to be executed also includes: receipt number, order number, unit name, contract number, project name, production time, pouring method, concrete type, mixer truck number, shipment quantity, receipt quantity, production quantity, mixer number, pouring location, transportation distance, receipt status, contact person, etc.

[0101] When the target mixer truck has completed loading and leaves the mixing station for the target construction site, the target mixer truck is monitored for transportation.

[0102] S2 obtains the tank video information of the target mixer truck;

[0103] The tank video information includes: tank side image and tank tail image.

[0104] The tank side image includes: a tank left side image, and / or a tank right side image.

[0105] The left side image of the tank shows a fixed position video stream of the left side of the tank body of the target mixer truck. In one embodiment of the present specification, a camera is installed below the left rearview mirror in front of the mixer truck as a left side camera to collect the left side image of the tank. Figure 2 As shown, in the image on the left side of the tank, the left side of the tank body of the mixer truck can be clearly seen, that is, the viewing angle of the left camera completely covers the visible part of the tank body.

[0106] The right side image of the tank shows a fixed position video stream of the left side of the tank body of the target mixer truck. In one embodiment of the present specification, a camera is installed below the right rearview mirror in front of the mixer truck as a right side camera for collecting the right side image of the tank. Figure 3 As shown, in the image on the right side of the tank, the right side of the tank body of the mixer truck can be clearly seen, that is, the right camera angle of view completely covers the visible part of the tank body.

[0107] The tank tail image shows a fixed position video stream of the bottom of the tank body of the target mixer truck. In one embodiment of the present specification, the camera is installed at the tail of the tank body of the mixer truck and as close to the top as possible, as a tail camera, for collecting the tank tail image. Figure 4 As shown, the rear camera is directed toward the ground, and the viewing angle of the rear camera covers the bottom of the tank and the position where the concrete is poured.

[0108] S3 detects the tank side image through a rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck changes;

[0109] The rotation directions include: no rotation, clockwise rotation, and counterclockwise rotation.

[0110] Herein, no rotation means that the tank body does not rotate clockwise or counterclockwise from the direction from the head of the tank body to the tail of the tank body.

[0111] Clockwise rotation means that the tank body rotates clockwise from the head of the tank body to the tail of the tank body.

[0112] Counterclockwise rotation means that the tank body rotates counterclockwise from the head of the tank body to the tail of the tank body.

[0113] like Figure 5 As shown, step S3 specifically includes:

[0114] S31 obtains the current tank side frame image of the tank side image;

[0115] The tank side image is a video stream or image sequence including the edge of the tank body. Each frame image of the tank side image is acquired in sequence, and the current frame image of the tank side image is used as the current tank side frame image.

[0116] S32: performing key point recognition on the current tank side frame image to obtain current features of the key points;

[0117] S321 performs a first preprocessing operation on the current tank side frame image to obtain the current tank side frame image after edge detection;

[0118] S321-1 performs a first grayscale processing on the current tank side frame image to obtain a grayscale current tank side frame image;

[0119] A grayscale algorithm is used to convert color images into grayscale images to reduce the amount of calculation and remove the influence of color information on detection.

[0120] S321-2 performs smoothing processing on the grayscale current tank side frame image to obtain a smoothed current tank side frame image;

[0121] A Gaussian blur filter is applied to smooth the grayscale current tank side frame image to reduce noise and details and make the edge smoother, which is beneficial to subsequent edge detection.

[0122] S321-3 performs edge detection on the smoothed current tank side frame image to determine the overall outline of the tank body and generate the current tank side frame image after edge detection.

[0123] The smoothed current tank side frame image is processed using an edge detection algorithm (such as Canny edge detection), edge points in the image are identified, and a clear edge between the tank body and the background is obtained.

[0124] S322 performs graphic detection on the current tank side frame image after the edge detection to extract key lines of the tank body;

[0125] The key lines include at least one of contour lines and boundary lines.

[0126] The contour line refers to the boundary line between the edge of the tank and the background. In one embodiment of the present specification, the contour line includes a plurality of straight line segments connected end to end.

[0127] The dividing line is located on the outer surface of the tank body, is the seam line of the tank body, and is parallel to the axis.

[0128] In one embodiment of the present specification, a Hough transform is performed on the current tank side frame image after edge detection to extract the key lines of the tank body. Specifically, S322-1 converts the current tank side frame image after edge detection into a binary image;

[0129] The current tank side frame image after edge detection is binarized, that is, the pixels in the image are divided into foreground (edge ​​points) and background.

[0130] S322-2 uses Hough transform to detect key lines of the tank.

[0131] By setting appropriate parameters (such as The accuracy, accuracy, threshold, etc.) to ensure that the boundary line of the tank can be detected.

[0132] S322-3 Find the unit with the largest accumulator value in the parameter space and assign the corresponding and The value is used as the parameter of the straight line segment. The detected straight line segment is mapped back to the original image space and marked or displayed.

[0133] S323 combines the key lines of the tank body to construct current features of the key points.

[0134] S323-1 Find key points;

[0135] The key points are pre-set and are points in the image with obvious local features and easy to identify.

[0136] S323-2 extracting motion information of key points of the current tank side frame image and the previous tank side frame image as current features of the key points;

[0137] The current feature of the key point may be the current displacement vector of the key point, which includes: the movement direction of the key point and the movement distance of the key point.

[0138] S33 combines the current feature of the key point and the previous feature of the key point to determine whether the rotation direction of the tank body has changed.

[0139] When the mixer truck is unloading, the rotation direction of the tank body will be reversed. Therefore, when conducting transportation monitoring to determine whether abnormal unloading occurs, it is necessary to first determine whether the rotation direction of the tank body has changed. At this time, it is necessary to combine two consecutive displacement vectors to determine whether the tank body is reversed in the current tank side frame image.

[0140] In one embodiment of the present specification, an optical flow method (such as Lucas-Kanade optical flow method) is used to track the key points on the edge of the tank body, and the displacement vector of each key point, that is, the moving direction and distance from the previous frame to the current frame, is calculated. By analyzing the displacement vector of the key point.

[0141] By analyzing the displacement vectors of key points, the overall rotation trend of the tank can be inferred. If the directions of two consecutive displacement vectors in the same tank side image are roughly the same and point to the same direction, it can be judged that the tank is rotating in that direction and there is no reversal. If the directions of two consecutive displacement vectors in the same tank side image are different, it is judged that the tank is reversed and the rotation direction of the tank has changed.

[0142] In one embodiment of the present specification, the rotation direction of the tank body is determined by combining the left side image and the right side image of the tank.

[0143] Specifically, based on the optical flow results of the two cameras, the average displacement directions of the pixels of the key points are compared.

[0144] Considering that after the camera is installed on the mixer truck, the camera position is basically fixed and will not move at will. In order to better judge the direction of the tank, the tank is divided by combining the tank images from the left camera and / or the right camera;

[0145] In one embodiment of the present specification, in the left camera, find the midpoint of the arc at the tail of the tank body; find the midpoint of the head of the tank body; use the line connecting the midpoint of the head of the tank body in the left camera and the midpoint of the arc at the tail of the tank body as the left center line; based on the left center line, divide the tank body in the left camera into two parts, i.e., the upper left part and the lower left part.

[0146] Similarly, in the right camera, find the midpoint of the arc at the tail of the tank; find the midpoint of the head of the tank; use the line connecting the midpoint of the head of the tank in the right camera and the midpoint of the arc at the tail of the tank as the right midline; based on the right midline, divide the tank in the right camera into two parts, i.e., the upper right part and the lower right part. If the left camera shows that the key point of the upper left part of the tank moves to the right, while the right camera shows that the key point of the lower right part of the tank moves to the left, or the left camera shows that the key point of the lower left part of the tank moves to the left, while the right camera shows that the key point of the upper right part of the tank moves to the right, then the tank rotates clockwise.

[0147] If the left camera shows that the key points on the upper left side of the tank move to the left, while the right camera shows that the key points on the lower right side of the tank move to the right, or if the left camera shows that the key points on the lower left side of the tank move to the right, while the right camera shows that the key points on the upper right side of the tank move to the left, then the tank rotates counterclockwise.

[0148] The direction of rotation is determined by calculating the sign of the mean displacement.

[0149] ;

[0150] By comparing the information from the two cameras, the error that may be caused by a single camera can be reduced and the accuracy can be improved. Moreover, the two cameras can observe the tank from different angles, which helps to have a more comprehensive understanding of the status of the tank and provides better viewing angle coverage.

[0151] In some cases, if only one camera is used (such as the left camera or the right camera), the rotation direction can also be determined by analyzing the displacement vector of the key points. However, this method may be limited by the camera installation position, angle, and field of view, and the accuracy may be slightly lower.

[0152] Of course, when one of the left camera or the right camera fails or is blocked, the other camera can still continue to work, ensuring the continuous operation of the system, enhancing robustness, and achieving fault redundancy.

[0153] S332-4 takes the current tank side frame image as the previous tank side frame image and performs the above operations on the new current tank side frame image to facilitate continuous monitoring.

[0154] S4: when the rotation direction of the tank body changes, a slump feature is identified from the tank tail image by using a concrete recognition model, and whether the concrete in the target mixer truck has been poured out is determined according to the slump feature;

[0155] like Figure 6 As shown, it specifically includes:

[0156] S41 obtains the current can tail frame image of the can tail image;

[0157] The can tail image is a video stream or image sequence containing the edge of the can body. Each frame image of the can tail image is acquired in sequence, and the current frame image of the can tail image is used as the current can tail frame image.

[0158] S42 performs concrete recognition on the current tank tail frame image to extract the slump characteristics of the target area;

[0159] The target area is the bottom of the tank and the area immediately below it.

[0160] S421 performs a second preprocessing operation on the current can tail frame image to obtain a preprocessed current can tail frame image;

[0161] S421-1 performs a second grayscale processing on the current can tail frame image to obtain a grayscaled current can tail frame image;

[0162] Through the above operations, the computational complexity is reduced and the structural information of the image is preserved.

[0163] S421-2: removing the static background in the grayscale current can tail frame image to obtain the subtracted current can tail frame image;

[0164] In one embodiment of the present specification, the static background in the current tank tail frame image is removed by background subtraction technology (such as Gaussian mixture model, MOG2, etc.).

[0165] S421-3 performs color adjustment on the subtracted current tank tail frame image to obtain the preprocessed current tank tail frame image;

[0166] The subtracted image is processed using threshold segmentation / color space transformation. Specifically, the image is converted from grayscale space or RGB space to a color space (such as HSV) that is more suitable for segmentation.

[0167] According to the color characteristics of concrete, a suitable threshold is set to determine and extract the region of interest (ROI), i.e. the target area.

[0168] Among them, the threshold is adjusted according to the color characteristics of concrete to ensure accurate segmentation.

[0169] S422 performs feature extraction on the preprocessed current tank tail frame image to obtain the slump feature of the target area.

[0170] S422-1 build training set;

[0171] A number of original images are acquired; the original images involve a process of pouring concrete.

[0172] The state information of the concrete in each frame of the original image is obtained; the state information includes: the flow speed and moving distance of the concrete, and the maximum width, maximum height and area of ​​the concrete area.

[0173] In one embodiment of the present specification, the difference of the concrete area between two adjacent frames is calculated by the inter-frame difference method to obtain the moving distance of the concrete. The motion vector of the concrete is calculated by the optical flow method to obtain the flow speed of the concrete. The maximum width and maximum height of the concrete area in each frame are found, and the area of ​​the concrete area in each frame is calculated by combining the maximum width and the maximum height.

[0174] Based on the manual annotation results, the slump of concrete in the original image is obtained.

[0175] Combine the state information and slump of concrete in the original image to construct training samples;

[0176] Obtain the training samples corresponding to each original image and construct a training set.

[0177] S422-2 build and train a concrete recognition model;

[0178] The concrete identification model is preferably a regression model.

[0179] As a preferred embodiment, in the concrete identification model, the slump .in, , , , , , is the regression coefficient; Indicates the flow velocity of concrete; Indicates the maximum width of the concrete area; Indicates the maximum height of the concrete area; The symbol indicates the distance the concrete is moved; Indicates the area of ​​the concrete area.

[0180] Substitute the training set into the concrete recognition model to determine the optimal value of each regression coefficient. In one embodiment of the present specification, the optimal regression coefficient is solved based on minimizing the mean square error. That is, .

[0181] Among them, the latest concrete recognition model 。

[0182] S423 performs feature extraction on the preprocessed current tank tail frame image to determine whether the concrete has been poured out;

[0183] S423-1, combining the pre-processed current tank tail frame image, analyzing whether the target area has dynamic changes;

[0184] In one embodiment of the present specification, a background model is constructed and updated to adapt to environmental factors such as lighting changes and slight vibrations, so as to accurately distinguish between the foreground (tank and possible concrete flow) and the background (stationary ground or equipment).

[0185] Compare the preprocessed current tank tail frame image with the background model to detect whether there are dynamic changes in the target area, so as to later determine whether these changes indicate the flow of concrete or the movement of the tank body.

[0186] Among them, dynamic changes refer to changes in color, brightness, texture, etc. of the target area.

[0187] S423-2: when a dynamic change occurs, detecting whether there is a target object under the tank;

[0188] The target object refers to the object having the concrete color characteristics.

[0189] In one embodiment of the present specification, a background subtraction technique or an inter-frame difference method is used to segment a foreground object from a pre-processed current tank tail frame image.

[0190] Perform color analysis on the foreground object and extract pixels or areas with the color of concrete.

[0191] Track the changes of foreground objects in consecutive frames and detect whether the target object appears.

[0192] In one embodiment of the present specification, by selecting concrete images under different weather conditions and different lighting conditions for training, the accuracy of the algorithm under different weather conditions is evaluated, the stability of the algorithm under different lighting conditions is tested, and the influence of factors such as concrete color changes and environmental occlusion are considered.

[0193] S423-3 When the target object appears, the flow morphology of the target object is identified to obtain the slump characteristics.

[0194] The optical flow method is applied to track the movement in the target area and calculate the expansion speed of the target object in the target area. Specifically, the state information of the target concrete is generated according to the expansion speed and shape change;

[0195] Status information includes: concrete flow rate , Maximum width of the concrete area , Maximum height of concrete area , Concrete moving distance , the area of ​​the concrete area .

[0196] S43 determines whether the concrete has been poured out by combining the slump characteristics and the preset slump conditions.

[0197] If the slump characteristics of the target object in the target area meet the preset slump conditions, it is determined that there are obvious signs of concrete flow in the target area and that the concrete has been poured out.

[0198] If the slump characteristics of the target object in the target area do not meet the preset slump conditions, it is determined that no obvious signs of concrete flow are detected in the target area, and it is judged that the concrete has not been poured out, and the next frame of image is captured for identification.

[0199] The preset slump condition is configured according to actual needs. In one embodiment of the present specification, the preset slump condition includes: slump characteristic ≠0.

[0200] In this specification, the transportation and unloading process of the mixer truck is recorded through video information and model analysis to achieve effective monitoring of the concrete transportation process and provide strong support for data analysis and tracing. In the present invention, the images of the entire transportation monitoring stage are effectively saved, reducing the driver's violations and improving the transparency of the transportation process; and, when violations occur, evidence can be obtained in a timely manner so that corresponding management measures can be taken.

[0201] S5: When it is detected that the concrete in the target mixer truck is poured out, based on the consistency between the actual position of the target mixer truck and the target position, it is determined whether the target mixer truck is unloading normally, and an early warning is issued according to the determination result.

[0202] Get the real-time geographic location of the mixer truck as the actual location of the mixer truck;

[0203] If the actual position of the mixer truck is consistent with the target position, it is considered to be normal unloading.

[0204] If the actual position of the mixer truck is inconsistent with the target position, it is considered that the unloading is not normal and the unloading behavior is abnormal, and an early warning is issued in time. Specifically, the tank side image, tank tail image, actual position of the mixer truck, unloading time, etc. can be recorded and sent to the dispatcher.

[0205] In this manual, by monitoring the working status of the mixer truck tank, the rotation direction of the tank and the flow rate of the concrete are used to determine whether it is in the unloading state. By verifying whether the actual position of the mixer truck is the planned target position, it is determined whether it is normal unloading. By real-time monitoring of the transportation status of the mixer truck, potential safety hazards can be discovered and resolved in a timely manner, such as vehicle failure, illegal resale or waste of concrete, etc., to ensure the safety of the transportation process and protect the economic interests of the enterprise.

[0206] Figure 7 A schematic diagram of a transport monitoring system for a mixer truck provided in an embodiment of this specification, the system comprising:

[0207] A task construction module 710 is used to construct a task to be executed, wherein the task to be executed includes: a target mixer truck and a target location;

[0208] The video acquisition module 720 is used to acquire the tank video information of the target mixer truck; the tank video information includes: a tank side image and a tank tail image;

[0209] A rotation detection module 730 is used to detect the tank side image through a rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck has changed;

[0210] A slump detection module 740 is used to identify the slump characteristics from the tank tail image through a concrete recognition model when the rotation direction of the tank body changes, and determine whether the concrete in the target mixer truck has been poured out according to the slump characteristics;

[0211] The position detection module 750 is used to determine whether the target mixer truck is unloading normally based on the consistency between the actual position of the target mixer truck and the target position when it is detected that the concrete in the target mixer truck is poured out, and to issue an early warning according to the judgment result.

[0212] Optionally, the rotation detection module 730 includes:

[0213] A tank side frame image acquisition submodule, used for acquiring a current tank side frame image of the tank side image;

[0214] A key point recognition submodule is used to perform key point recognition on the current tank side frame image to obtain current features of the key points;

[0215] The rotation judgment submodule is used to judge whether the rotation direction of the tank body has changed by combining the current feature of the key point and the previous feature of the key point.

[0216] Optionally, the key point identification submodule includes:

[0217] A first preprocessing unit, configured to perform a first preprocessing operation on the current tank side frame image to obtain the current tank side frame image after edge detection;

[0218] A graphics detection unit, used to perform graphics detection on the current tank side frame image after the edge detection, and extract key lines of the tank body;

[0219] The feature construction unit is used to construct the current features of the key points in combination with the key lines of the tank body.

[0220] Optionally, the first preprocessing unit includes:

[0221] A first grayscale subunit, configured to perform a first grayscale process on the current tank side frame image to obtain a grayscaled current tank side frame image;

[0222] A smoothing processing subunit, used for performing smoothing processing on the grayscale current tank side frame image to obtain a smoothed current tank side frame image;

[0223] The edge detection subunit is used to perform edge detection on the smoothed current tank side frame image, determine the overall contour of the tank body, and generate the current tank side frame image after edge detection.

[0224] Optionally, the slump detection module 740 includes:

[0225] A can tail frame image acquisition submodule is used to acquire a current can tail frame image of the can tail image;

[0226] A concrete recognition submodule, used to perform concrete recognition on the current tank tail frame image and extract the slump characteristics of the target area;

[0227] The preset feature judgment submodule is used to judge whether the concrete has been poured out by combining the slump feature and the preset slump condition.

[0228] Optionally, the concrete identification submodule includes:

[0229] A second preprocessing unit, configured to perform a second preprocessing operation on the current can tail frame image to obtain a preprocessed current can tail frame image;

[0230] The slump determination unit is used to extract features from the preprocessed current can tail frame image to obtain the slump features of the target area.

[0231] Optionally, the slump determination unit comprises:

[0232] A dynamic change analysis subunit, used to analyze whether there is a dynamic change in the target area in combination with the pre-processed current tank tail frame image;

[0233] A target object detection subunit, used for detecting whether there is a target object under the tank body when dynamic changes occur;

[0234] The slump feature generating subunit is used to identify the flow form of the target object and generate the slump feature when the target object appears.

[0235] The functions of the system of the embodiment of the present invention have been described in the above method embodiment, so for details not fully described in this embodiment, please refer to the relevant description in the above embodiment, and no further description will be given here.

[0236] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0237] The electronic device embodiment of the present invention is described below, and the electronic device can be regarded as a physical implementation of the method and device embodiments of the present invention described above. The details described in the electronic device embodiment of the present invention should be regarded as a supplement to the above method or device embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above method or device embodiments.

[0238] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Figure 8 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0239] like Figure 8 As shown, the computer device 800 of this exemplary embodiment is in the form of a general data processing device. The components of the computer device 800 may include but are not limited to: at least one processor 810, at least one memory 820, a network interface 830, a display unit 840, an input component 850, etc.

[0240] The memory 820 stores a computer-readable program, which may be a source program or a code of a read-only program. The program may be executed by the processor 810, so that the processor 810 performs the steps of various embodiments of the present invention. For example, the processor 810 may perform the following steps: Figure 1 Steps shown.

[0241] The memory 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit, and may further include a read-only memory unit (ROM). The memory 820 may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment.

[0242] Also included is a bus (not shown) which may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0243] The computer device 800 may also communicate with one or more external devices (e.g., keyboard, display, network device, Bluetooth device, etc.) so that a user can interact with the computer device 800 via these external devices, and / or so that the computer device 800 can communicate with one or more other data processing devices (e.g., routers, modems, etc.). Such communication may be performed through the network interface 830, and may also be performed through a network adapter with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet). The network adapter may communicate with other modules of the computer device 800 via a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in the computer device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0244] Fig. 9 Schematic diagram of a computer readable medium embodiment of the present invention. Fig. 9 As shown, the computer program can be stored on one or more computer-readable media. The computer-readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. When the computer program is executed by one or more data processing devices, the computer-readable medium is enabled to implement the above method of the present invention.

[0245] Through the description of the above implementation modes, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation mode of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a data processing device (which can be a personal computer, a server, or a network device, etc.) to perform the above method according to the present invention.

[0246] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0247] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0248] In summary, the present invention can be implemented by a method, apparatus, electronic device or computer-readable medium that executes a computer program. In practice, a general data processing device such as a microprocessor or a digital signal processor (DSP) can be used to implement some or all functions of the present invention.

[0249] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring the transportation of a mixer truck, characterized in that: include: Construct tasks to be performed. Specifically, determine the total concrete production volume and target location according to the contract and related projects, and create task orders; Combine the task list with the current capacity of the mixer truck to schedule and determine each target mixer truck; Sending corresponding tasks to be executed to each of the target mixer trucks; The tasks to be performed include: a target mixer truck and a target location; Obtaining the tank video information of the target mixer truck; the tank video information includes: a tank side image and a tank tail image; The tank side image is detected by a rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck has changed; When the rotation direction of the tank body changes, the slump characteristics are identified from the tank tail image through the concrete recognition model, and whether the concrete in the target mixer truck has been poured out is judged according to the slump characteristics; specifically, the target area is determined according to the color characteristics of the concrete; whether the target area has dynamic changes is detected; when the dynamic changes occur, whether there is a target object under the tank body is detected; when the target object appears, the movement in the target area is tracked, the expansion speed of the target object in the target area is calculated, and the state information of the target concrete is generated according to the expansion speed and shape change to obtain the slump characteristics; wherein the state information includes: the flow speed of the concrete , Maximum width of the concrete area , Maximum height of concrete area , Concrete moving distance , the area of ​​the concrete area ; Slump ; , , , , , is the regression coefficient; When it is detected that the concrete in the target mixer truck is poured out, it is judged whether the target mixer truck is unloading normally based on the consistency between the actual position of the target mixer truck and the target position, and an early warning is issued according to the judgment result.

2. A method for monitoring the transportation of a mixer truck according to claim 1, characterized in that: The detecting the tank side image by the rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck has changed includes: Acquire a current tank side frame image of the tank side image; Performing key point recognition on the current tank side frame image to obtain current features of the key points; Combining the current feature of the key point with the previous feature of the key point, it is determined whether the rotation direction of the tank body has changed.

3. A method for monitoring the transportation of a mixer truck according to claim 2, characterized in that: The step of performing key point recognition on the current tank side frame image to obtain current features of the key points includes: Performing a first preprocessing operation on the current tank side frame image to obtain the current tank side frame image after edge detection; Performing graphic detection on the current tank side frame image after the edge detection to extract key lines of the tank body; Combined with the key lines of the tank body, the current features of the key points are constructed.

4. A method for monitoring the transportation of a mixer truck according to claim 3, characterized in that: The performing a first preprocessing operation on the current tank side frame image to obtain the current tank side frame image after edge detection includes: Performing a first grayscale processing on the current tank side frame image to obtain a grayscaled current tank side frame image; Smoothing the grayscaled current tank side frame image to obtain a smoothed current tank side frame image; The smoothed current tank side frame image is subjected to edge detection to determine the overall contour of the tank body, thereby generating the current tank side frame image after edge detection.

5. The method for monitoring the transportation of a mixer truck according to claim 1, characterized in that: When the rotation direction of the tank body changes, the slump feature is identified from the tank tail image by using a concrete recognition model, and whether the concrete in the target mixer truck is poured out is determined according to the slump feature, including: Obtaining a current can tail frame image of the can tail image; Performing concrete recognition on the current tank tail frame image to extract slump characteristics of the target area; In combination with the slump characteristics and the preset slump conditions, it is determined whether the concrete has been poured out.

6. A method for monitoring the transportation of a mixer truck according to claim 5, characterized in that: The step of performing concrete recognition on the current tank tail frame image and extracting the slump characteristics of the target area includes: Performing a second preprocessing operation on the current can tail frame image to obtain a preprocessed current can tail frame image; Feature extraction is performed on the preprocessed current tank tail frame image to obtain the slump feature of the target area.

7. A method for monitoring the transportation of a mixer truck according to claim 6, characterized in that: The step of extracting features from the preprocessed current tank tail frame image to obtain the slump features of the target area includes: In combination with the pre-processed current tank tail frame image, analyzing whether the target area has dynamic changes; When dynamic changes occur, detecting whether there is a target object under the tank; When the target object appears, the flow morphology of the target object is identified and the slump feature is generated.

8. A transport monitoring system for a mixer truck, characterized in that: include: The task construction module is used to construct tasks to be executed. Specifically, according to the contract and related projects, the total concrete production volume and target location are determined, and a task order is created; the task order and the current capacity of the mixer truck are combined for scheduling, and each target mixer truck is determined; Sending corresponding tasks to be executed to each of the target mixer trucks; The tasks to be performed include: a target mixer truck and a target location; A video acquisition module is used to acquire the tank video information of the target mixer truck; the tank video information includes: a tank side image and a tank tail image; A rotation detection module, used to detect the tank side image through a rotation detection model to determine whether the rotation direction of the tank body of the target mixer truck has changed; The slump detection module is used to identify the slump characteristics from the tank tail image through the concrete recognition model when the rotation direction of the tank body changes, and judge whether the concrete in the target mixer truck has been poured out according to the slump characteristics; specifically, determine the target area according to the color characteristics of the concrete; detect whether there is a dynamic change in the target area; when the dynamic change occurs, detect whether there is a target object under the tank body; when the target object appears, track the movement in the target area, calculate the expansion speed of the target object in the target area, generate the state information of the target concrete according to the expansion speed and shape change, and obtain the slump characteristics; wherein the state information includes: the flow speed of the concrete , Maximum width of the concrete area , Maximum height of concrete area , Concrete moving distance , the area of ​​the concrete area ; Slump ; , , , , , is the regression coefficient; The position detection module is used to determine whether the target mixer truck is unloading normally based on the consistency between the actual position of the target mixer truck and the target position when it is detected that the concrete in the target mixer truck is poured out, and to issue an early warning according to the judgment result.

9. An electronic device, wherein: The electronic device includes: processor; and, A memory storing computer executable instructions which, when executed, cause the processor to perform a method according to any one of claims 1-7.

10. A computer-readable storage medium, wherein: The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method of any one of claims 1 to 7 is implemented.

Images