A train cargo box measurement system and method based on laser radar

Abstract

The present invention discloses a method and system for measuring a train cargo box based on laser radar. When a train cargo box is transporting coal, unusual stacking shapes may occur, and it is necessary to determine whether it is the transported coal or other objects fixed in the cargo box; therefore, the position of the coal in space is detected to solve the above problem, thereby calculating the volume of the coal; according to the frequency of the swing of the cargo box during movement, the backbone straight line of the object is found; according to different weights, the backbone straight lines of different objects are matched; and by determining that the inclination angles of two overlapping coals during movement are different, while the inclination angles of a whole piece of coal during movement are the same, and the objects fixed in the train cargo box will not tilt; therefore, according to the above, the positions of the stacked coal at multiple angles are found, thereby constructing a three-dimensional image of the stacked coal, and calculating the volume of the stacked coal.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a laser radar-based train cargo box measurement method and system. Background Art

[0002] In the modern railway transportation industry, effective monitoring of train cargo compartments is crucial to improving transportation efficiency and ensuring transportation safety. At present, common train cargo compartment monitoring technologies mainly include RFID-based identification systems and simple video monitoring systems. These systems can provide basic cargo compartment identification functions and visual monitoring, but there are several obvious disadvantages. Limited monitoring functions: Existing systems focus more on the identification and location tracking of cargo compartments, while the real-time status monitoring of cargo (such as weight changes, cargo type identification, loading conditions, etc.) is insufficient, resulting in the inability to effectively monitor the safety and integrity of cargo during transportation. Data processing delay: Existing technologies have large delays in data processing, especially when processing large amounts of video data, with poor real-time performance and inability to provide instant feedback and processing, which affects the timeliness and accuracy of decision-making. Poor system integration: The lack of effective integration between different monitoring equipment and systems has led to serious information islands between systems, low data sharing and comprehensive utilization, increased operating costs, and reduced monitoring efficiency. Poor environmental adaptability: Traditional video monitoring systems rely heavily on environmental conditions. For example, at night or in severe weather conditions, video quality deteriorates, making it difficult to accurately identify and monitor the status of cargo compartments.

[0003] When a train cargo box is transporting objects, there may be multiple objects stacked in unusual shapes. It is not determined whether they are transported objects or other objects fixed in the cargo box, which leads to detection errors and incorrect calculation of the volume of the goods. Summary of the invention

[0004] The purpose of the present invention is to provide a laser radar-based train cargo box measurement method and system to solve the above-mentioned problems existing in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a method for measuring a train cargo box based on a laser radar, comprising:

[0006] The laser three-dimensional data, a steady cargo image set and a sudden cargo image set at multiple time points are obtained; the laser three-dimensional data includes the distance from each position of the stacked coal in the cargo box to the laser device; the steady cargo image set includes a plurality of steady cargo images; the plurality of steady cargo images represent images containing stacked coal taken at multiple angles when the train cargo box is at a constant speed; the sudden cargo image set includes a plurality of sudden cargo images; the plurality of sudden cargo images represent images containing stacked coal taken at multiple angles after the train cargo box changes speed;

[0007] Based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, a plurality of special segmentation boundary area images are obtained according to the position and color of the coal stack; the special segmentation boundary area images include a plurality of special segmentation boundary areas;

[0008] The t*m special segmentation boundary region images corresponding to the stable cargo image set at t time points are used as stable special boundary region images; the t*n special segmentation boundary region images corresponding to the sudden cargo image set at t time points are used as motion special boundary region images;

[0009] Based on the stable special boundary region image, the backbone position of the coal stack is found to obtain the backbone image of the object; multiple stable special boundary region images correspond to obtaining multiple backbone images of the object;

[0010] Based on the motion special image and the corresponding multiple object backbone images, the different tilt angles of different objects in the motion process are judged to obtain a detected special image; the detected special image contains multiple locations of stacked coal that are difficult to detect by target detection; multiple object backbone images correspond to multiple detected special images;

[0011] Based on multiple detected special images, the volume of the coal in the cargo box is detected to obtain the volume of the coal.

[0012] Optionally, based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, a plurality of special segmentation boundary area images are obtained according to the position and color of the coal stack, including:

[0013] Performing edge detection on the mutant cargo image in the mutant cargo image set or the stable cargo image in the stable cargo image set to obtain an edge image; the edge image includes a plurality of edge closed regions;

[0014] Obtain n+m edge images corresponding to the n mutant cargo images in the mutant cargo image set and the m stable cargo images in the stable cargo image set;

[0015] According to the laser three-dimensional data, the distance of the position is determined to obtain a depth distance image; the depth distance image includes a plurality of position areas with different depths; the depth distance image represents the distance from each position in the carriage to the laser device;

[0016] Segmenting the depth range image according to the positions where the distance mutation occurs to obtain a plurality of position areas;

[0017] Based on the depth range image including the multiple position areas and the multiple edge images, position and color matching is performed to obtain multiple special segmentation boundary area images.

[0018] Optionally, the step of finding the backbone position of the coal stack based on the stable special boundary region image to obtain the backbone image of the object includes:

[0019] Based on the stable special boundary region image, a stable center point set is obtained; the stable center point set includes multiple stable center points; multiple special segmentation boundary regions of the stable special boundary region image correspond to multiple stable center point sets;

[0020] Fitting the plurality of stable center points into highly parallel straight lines that are perpendicular to the stable special boundary region image to obtain a backbone straight line of the object;

[0021] A plurality of object backbone straight lines are obtained corresponding to a plurality of stable special boundary regions of the stable special boundary region image;

[0022] The plurality of object backbone straight lines are marked in the stable special boundary region image to obtain an object backbone image.

[0023] Optionally, the obtaining of the detected special image based on the motion special image and the corresponding plurality of object backbone images is judged according to different tilt angles of different objects in the motion process, including:

[0024] Tracking multiple stable center points corresponding to the object backbone straight line in the object backbone image, finding the moving position of the object backbone straight line in the motion special boundary area image according to the change of the object shape, and obtaining a tilted backbone position set; the tilted backbone position set includes multiple tilted backbone positions; the object backbone image and the motion special boundary area image are images taken by the same camera device at different time points;

[0025] Obtaining a plurality of inclined backbone position sets corresponding to a plurality of object backbone straight lines in the object backbone image;

[0026] Fitting a plurality of inclined backbone position sets into an inclined backbone straight line;

[0027] The angle between the inclined backbone straight line and the edge corresponding to the height of the object backbone image is taken as the inclination angle;

[0028] If the edge distance between two special segmentation boundary areas in the object skeleton image is less than the edge distance threshold and the inclination angles are equal, the two special segmentation boundary areas are marked as a whole object to obtain multiple special detection object positions;

[0029] The positions of multiple special detection objects are marked in the object backbone image to obtain a detected special image.

[0030] Optionally, obtaining a stable center point set based on the stable special boundary area image includes:

[0031] A coordinate axis is established according to the stable special boundary region image; the width of the stable special boundary region image corresponds to the abscissa; the height of the stable special boundary region image corresponds to the ordinate; and the lower left corner of the stable special boundary region image corresponds to the coordinate origin;

[0032] Acquire a range of a vertical coordinate of a smooth special boundary region corresponding to the smooth special boundary region image as a smooth special boundary height range;

[0033] Add two positions with the same ordinate in the stable special boundary region in the stable special boundary height range to the same height boundary position set; obtain multiple same height boundary position sets corresponding to multiple ordinates;

[0034] The horizontal coordinates corresponding to the two positions in the same-height boundary position set are used as the horizontal coordinates of the same-height edge;

[0035] The horizontal coordinates of two edges of the same height in the set of edge positions of the same height are averaged to obtain a stable center point; the vertical coordinate of the stable center point is the vertical coordinate of the edge position of the same height corresponding to the horizontal coordinate of the edge of the same height;

[0036] Multiple stable center points are obtained corresponding to multiple sets of boundary positions at the same height;

[0037] The multiple stable center points are added to a stable center point set.

[0038] Optionally, the matching of positions and colors based on the depth range image containing the multiple position areas and the multiple edge images to obtain multiple special segmentation boundary area images includes:

[0039] Obtaining a camera angle; the camera angle represents an angle between a depth range image and an edge image;

[0040] According to the camera angle, the position of the position area corresponding to the depth range image on the edge image is marked by a first convolutional network to obtain a marked position edge image;

[0041] A plurality of edge images corresponding to a plurality of marked position edge images are obtained;

[0042] According to the adjacent states of the camera devices corresponding to the marked position edge images, the marked position edge images at multiple angles are arranged in sequence to obtain a marked position edge image set;

[0043] The labels of the values ​​in the marked position edge image set are shifted backward by 1 bit each time to obtain multiple marked position edge image sets;

[0044] Input the set of edge images at the marked positions into the color convolutional network to detect the positions of similar patterns and obtain the position color features;

[0045] Multiple position color features are obtained corresponding to multiple marked position edge image sets;

[0046] The position color features are input into the position detection network to detect the position of the special object, and a special segmentation boundary area image containing multiple special segmentation boundary areas is obtained; multiple position color features correspond to multiple special segmentation boundary area images.

[0047] Optionally, the detecting the volume of the coal in the cargo box based on the plurality of detected special images to obtain the volume of the coal includes:

[0048] The values ​​of the areas where the special objects exist in the plurality of detected special images are set to 1, so as to obtain a detected special three-dimensional matrix;

[0049] Obtaining a detection volume; the detection volume represents the volume of the area corresponding to the entire image of the detected special three-dimensional image in the cargo box;

[0050] The detected special three-dimensional matrix and the corresponding detection volume are input into a three-dimensional detection network to detect the volume of coal in the cargo box to obtain the volume of coal.

[0051] Optionally, the laser three-dimensional data is acquired by a laser radar distributed on the top of the cargo compartment.

[0052] Optionally, the stable cargo image set and the sudden cargo image set are real-time images of multiple objects inside the cargo compartment taken by camera devices distributed around the cargo compartment.

[0053] In a second aspect, an embodiment of the present invention provides a laser radar-based train cargo box measurement system, comprising:

[0054] Acquisition module: obtains laser three-dimensional data, a stable cargo image set and a sudden change cargo image set at multiple time points; the laser three-dimensional data includes the distance from each position of the stacked coal in the cargo box to the laser device; the stable cargo image set includes multiple stable cargo images; the multiple stable cargo images represent images containing stacked coal taken at multiple angles when the train cargo box is at a constant speed; the sudden change cargo image set includes multiple sudden change cargo images; the multiple sudden change cargo images represent images containing stacked coal taken at multiple angles after the train cargo box changes speed;

[0055] Special object detection module: Based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, a plurality of special segmentation boundary area images are obtained according to the position and color of the coal stack; the special segmentation boundary area image contains a plurality of special segmentation boundary areas; the t*m special segmentation boundary area images corresponding to the stable cargo image set at t time points are used as stable special boundary area images; the t*n special segmentation boundary area images corresponding to the sudden cargo image set at t time points are used as motion special boundary area images; based on the stable special boundary area images, the backbone position of the coal stack is found to obtain the object backbone image; a plurality of stable special boundary area images correspond to a plurality of object backbone images; based on the motion special image and the corresponding plurality of object backbone images, different objects are judged according to different inclination angles during the motion process to obtain a detected special image; the detected special image contains the positions of the stacked coal that are difficult to detect by multiple target detections; a plurality of detected special images are obtained corresponding to the plurality of object backbone images;

[0056] Volume detection module: Based on multiple detected special images, the volume of coal in the cargo box is detected to obtain the volume of coal.

[0057] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:

[0058] An embodiment of the present invention also provides a laser radar-based train cargo box measurement method and system, the method comprising: obtaining laser three-dimensional data, a stable cargo image set and a sudden cargo image set at multiple time points; the laser three-dimensional data includes the distance from each position of the stacked coal in the cargo box to the laser device; the stable cargo image set includes multiple stable cargo images; the multiple stable cargo images represent images containing stacked coal taken at multiple angles when the train cargo box is at a constant speed; the sudden cargo image set includes multiple sudden cargo images; the multiple sudden cargo images represent images containing stacked coal taken at multiple angles after the train cargo box changes speed; based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, multiple special segmentation boundary area images are obtained according to the position and color of the coal stacking; the special segmentation boundary area images include multiple Special segmentation boundary area; t*m special segmentation boundary area images corresponding to the set of stable cargo images at t time points are used as stable special boundary area images; t*n special segmentation boundary area images corresponding to the set of sudden cargo images at t time points are used as motion special boundary area images; based on the stable special boundary area images, the backbone position of the coal stack is found to obtain the object backbone image; multiple stable special boundary area images correspond to multiple object backbone images; based on the motion special image and the corresponding multiple object backbone images, different objects are judged according to different inclination angles during the motion process to obtain a detected special image; the detected special image contains multiple locations of stacked coal that are difficult to detect by target detection; multiple object backbone images correspond to multiple detected special images; based on multiple detected special images, the volume of coal in the cargo box is detected to obtain the coal volume.

[0059] The present invention integrates high-precision laser radar and high-definition camera, and is supplemented by advanced image processing technology and deep learning algorithm, so as to accurately monitor the loading status of goods in real time, including the precise location, quantity and potential safety risks of goods. This is often impossible to achieve in the prior art, especially in dynamic environments and complex lighting conditions. Improve data processing speed and accuracy: Using deep learning algorithm, the present invention can automatically identify and classify the types of goods, reduce manual intervention, and significantly improve the speed and accuracy of data processing. Compared with the prior art, the present invention can effectively reduce false alarms and missed alarms, and improve the reliability of the overall monitoring system. High system integration, easy maintenance and upgrade: The system design of the present invention adopts modularization concept, and each component works together without interfering with each other, which makes the maintenance and upgrade of the system more convenient. For example, if any part of the camera or laser radar needs to be replaced or upgraded, it will not affect other parts of the system. Strong adaptability, stable operation in various environments: The present invention particularly optimizes the environmental adaptability of the system, and can still maintain efficient monitoring performance at night or in severe weather conditions. This feature solves the problem of performance degradation of the prior art under specific environmental conditions. Significant economic benefits: By improving cargo loading efficiency and reducing cargo losses, the present invention not only improves logistics efficiency, but also reduces potential economic losses, bringing significant economic benefits to the using companies.

[0060] In the present invention, when the train cargo box is transporting coal, unusual stacking shapes may occur, and it is necessary to determine whether it is the transported coal or other objects fixed in the cargo box. Therefore, the position of the coal in space is detected to solve the above problem, thereby calculating the volume of the coal. The backbone straight line of the object is found according to the frequency of the swing of the cargo box during the movement. The backbone straight lines of different objects are matched according to different weights. And by judging that the inclination angles of two overlapping coals during the movement are different, and the inclination angles of a whole piece of coal during the movement are the same, and the objects fixed in the train cargo box will not tilt. Therefore, according to the above, the positions of the stacked coal at multiple angles are found, thereby constructing a three-dimensional image of the stacked coal, and calculating the volume of the stacked coal.

[0061] In short, the present invention has effectively overcome many deficiencies of the prior art through technical innovation, brought about comprehensive performance improvement and economic benefits, and has great market application prospects. It has achieved the technical effect of not only being able to detect special objects that are difficult to detect, but also being able to judge the skeleton of the object under different accelerations, thereby more accurately calculating the volume of the stacked coal. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a flow chart of a train cargo box measurement method based on laser radar provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0064] Example 1

[0065] like Figure 1 As shown, an embodiment of the present invention provides a train cargo box measurement method based on laser radar, the method comprising:

[0066] S101: Obtain laser three-dimensional data, a set of stable cargo images and a set of sudden cargo images at multiple time points; the laser three-dimensional data includes the distance from each position of the stacked coal in the cargo box to the laser device; the stable cargo image set includes multiple stable cargo images; the multiple stable cargo images represent images containing stacked coal taken at multiple angles when the train cargo box is at a constant speed; the sudden cargo image set includes multiple sudden cargo images; the multiple sudden cargo images represent images containing stacked coal taken at multiple angles after the train cargo box changes speed.

[0067] For example, the stable cargo image is an image taken by the camera when the train cargo box is always at 120 km / h. The sudden cargo image is an image taken by the camera when the current time point is 140 km / h. The speed changes from 120 km / h to 140 km / h.

[0068] Among them, the images in the stable cargo image set and the sudden cargo image set are all clear images, and blurred images are removed.

[0069] The cargo image refers to an image containing the cargo taken by multiple camera devices in the cargo box.

[0070] For example, a camera device is provided on each of the four sides of the train cargo box, namely a first camera device, a second camera device, a third camera device and a fourth camera device.

[0071] S102: Based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, a plurality of special segmentation boundary area images are obtained according to the position and color of the coal stack; the special segmentation boundary area images include a plurality of special segmentation boundary areas.

[0072] S103: The t*m special segmentation boundary region images corresponding to the set of stable cargo images at t time points are used as stable special boundary region images. The t*n special segmentation boundary region images corresponding to the set of sudden cargo images at t time points are used as motion special boundary region images.

[0073] Wherein, t is a natural number greater than 0.

[0074] Among them, the mutant cargo image set includes n mutant cargo images and the stable cargo image set includes m stable cargo images.

[0075] The stable special boundary region image represents a special segmentation boundary region image at a constant speed, and the moving special boundary region image represents a special segmentation boundary region image at a variable speed.

[0076] S104: Based on the smooth special boundary region image, find the backbone position of the coal stack and obtain the object backbone image; multiple smooth special boundary region images correspond to obtaining multiple object backbone images.

[0077] S105: Based on the motion special image and the corresponding multiple object backbone images, different objects are judged according to different tilt angles during the motion process to obtain a detected special image. The detected special image contains multiple locations of stacked coal that are difficult to detect by target detection. Multiple object backbone images correspond to multiple detected special images.

[0078] S106: Based on the multiple detected special images, detect the volume of the coal in the cargo box to obtain the volume of the coal.

[0079] Optionally, based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, a plurality of special segmentation boundary area images are obtained according to the position and color of the coal stack, including:

[0080] Perform edge detection on the sudden change cargo image in the sudden change cargo image set or the stable cargo image in the stable cargo image set to obtain an edge image. The edge image includes a plurality of edge closed regions.

[0081] Among them, the candy algorithm is used for edge detection.

[0082] n+m edge images are obtained corresponding to the n mutant cargo images in the mutant cargo image set and the m stable cargo images in the stable cargo image set.

[0083] Wherein, n and m are natural numbers greater than 0.

[0084] According to the laser three-dimensional data, the distance of the position is determined to obtain a depth distance image. The depth distance image represents the distance from each position in the carriage to the laser device.

[0085] Among them, if a 1-meter-high box is placed, the distance between the box and the ground to the lidar can be detected to be different, so the box can be segmented.

[0086] The depth distance image represents a depth map.

[0087] The depth range image is segmented according to the positions where the distance mutation occurs to obtain a plurality of position areas.

[0088] The candy operator is used to calculate the edge of the depth distance image to determine whether the distance changes suddenly, and then the position segmentation is performed.

[0089] Based on the depth range image including the multiple position areas and the multiple edge images, position and color matching is performed to obtain multiple special segmentation boundary area images.

[0090] Optionally, the step of finding the backbone position of the coal stack based on the stable special boundary region image to obtain the backbone image of the object includes:

[0091] Based on the stable special boundary region image, a stable center point set is obtained. The stable center point set includes multiple stable center points. Multiple stable center point sets are obtained corresponding to multiple special segmentation boundary regions of the stable special boundary region image.

[0092] The stable center point represents the center point on multiple horizontal planes of a special segmentation boundary area.

[0093] The plurality of stable center points are fitted into highly parallel straight lines that are perpendicular to the stable special boundary region image to obtain the object backbone straight line.

[0094] Multiple object backbone straight lines are obtained corresponding to multiple smooth special boundary regions of the smooth special boundary region image.

[0095] The plurality of object backbone straight lines are marked in the stable special boundary region image to obtain an object backbone image.

[0096] Optionally, the obtaining of the detected special image based on the motion special image and the corresponding plurality of object backbone images is judged according to different tilt angles of different objects in the motion process, including:

[0097] The plurality of stable center points corresponding to the object backbone straight line in the object backbone image are tracked, and the moving position of the object backbone straight line in the motion special boundary region image is found according to the change of the object shape, so as to obtain a tilted backbone position set. The tilted backbone position set includes a plurality of tilted backbone positions. The object backbone image and the motion special boundary region image are images taken by the same camera device at different time points.

[0098] Among them, in this embodiment, a deep-sort multi-target tracking algorithm is used to track the backbone straight line of the object.

[0099] A plurality of object backbone straight lines in the object backbone image correspond to a plurality of inclined backbone position sets.

[0100] A plurality of sets of inclined backbone positions are fitted into an inclined backbone line.

[0101] In this embodiment, the least square method is used to fit multiple inclined backbone position sets into inclined backbone straight lines.

[0102] The included angle between the inclined backbone straight line and the acute angle corresponding to the height of the object backbone image is taken as the inclination angle.

[0103] The side corresponding to the height of the object backbone image is the ordinate of the established coordinate axis.

[0104] If the edge distance between two special segmentation boundary areas in the object skeleton image is less than the edge distance threshold and the inclination angles are equal, the two special segmentation boundary areas are marked as a whole object to obtain the position of the special detection object.

[0105] Among them, the whole object corresponds to a piece of coal in the stack.

[0106] The edge distance threshold is 1, and the unit is pixel. The edge distance is the minimum distance between two special segmentation boundary areas. By traversing the distances between all pixel points in the two special segmentation boundary areas, the distance with the minimum value is found as the edge distance.

[0107] The positions of multiple special detection objects are marked in the object backbone image to obtain a detected special image.

[0108] Optionally, obtaining a stable center point set based on the stable special boundary area image includes:

[0109] A coordinate axis is established according to the stable special boundary region image. The width of the stable special boundary region image corresponds to the horizontal coordinate. The height of the stable special boundary region image corresponds to the vertical coordinate. The lower left corner of the stable special boundary region image corresponds to the coordinate origin.

[0110] The range of the ordinate of the smooth special boundary area corresponding to the smooth special boundary area image is obtained as the smooth special boundary height range.

[0111] Among them, if the maximum value of the stable special boundary area in the vertical coordinate is 50 and the minimum value is 25, then the stable special boundary height range is [25,50].

[0112] Two positions with the same ordinate in the stable special boundary region in the stable special boundary height range are added to the same height boundary position set. Multiple ordinates correspond to multiple same height boundary position sets.

[0113] Among them, it is equivalent to a straight line parallel to the horizontal plane passing through the stable special boundary area, and two intersecting positions are obtained and added to the same height boundary position set. Multiple vertical coordinates correspond to multiple straight lines parallel to the horizontal plane, corresponding to multiple same height boundary position sets.

[0114] The horizontal coordinates corresponding to the two positions in the same-height boundary position set are used as the horizontal coordinates of the same-height edge.

[0115] The horizontal coordinates of two edges of the same height in the set of edge positions of the same height are averaged to obtain a stable center point. The vertical coordinate of the stable center point is the vertical coordinate of the edge position of the same height corresponding to the horizontal coordinate of the edge of the same height.

[0116] Multiple stable center points are obtained corresponding to multiple sets of boundary positions at the same height.

[0117] The multiple stable center points are added to a stable center point set.

[0118] Optionally, the matching of positions and colors based on the depth range image containing the multiple position areas and the multiple edge images to obtain multiple special segmentation boundary area images includes:

[0119] A camera angle is obtained, wherein the camera angle represents an angle between a depth range image and an edge image.

[0120] According to the camera angle, the position of the position area corresponding to the depth range image on the edge image is marked through the first convolutional network to obtain a marked position edge image.

[0121] Among them, the laser radar at the top finds the positions of different objects and segments them out. Because the laser radar is perpendicular to the camera device, it can segment the edge image into different widths based on the depth image, and each width represents the width of an object.

[0122] In this embodiment, the object represents stacked coal.

[0123] Among them, in this embodiment, the first convolutional network is a convolutional neural network (Convolutional Neural Networks, CNN).

[0124] Multiple edge images correspond to multiple marked position edge images.

[0125] The multiple edge images and the multiple mark position edge images represent images captured by camera devices at multiple angles.

[0126] According to the adjacent states of the camera devices corresponding to the marked position edge images, the marked position edge images at multiple angles are arranged in sequence to obtain a marked position edge image set.

[0127] Wherein, if four cameras are installed in a train cargo box, the marked position edge image taken by the camera near the front of the train cargo box is set as the first marked position edge image, the marked position edge image taken by the camera to the left of the camera near the front of the train cargo box is set as the second marked position edge image, the marked position edge image taken by the camera near the rear of the train cargo box is set as the third marked position edge image, and the marked position edge image taken by the camera to the right of the camera near the front of the train cargo box is set as the fourth marked position edge image. The arrangement in the marked position edge image set is {first marked position edge image, second marked position edge image, third marked position edge image, fourth marked position edge image}.

[0128] The labels of the values ​​in the marked position edge image set are shifted backward by 1 bit each time to obtain multiple marked position edge image sets.

[0129] If the label is the last digit, setting the label with the last digit as 1 to 0 is equivalent to moving the value with the last digit as the first digit. For example, the plurality of mark position edge image sets are {the fourth mark position edge image, the first mark position edge image, the second mark position edge image, the third mark position edge image}, {the third mark position edge image, the fourth mark position edge image, the first mark position edge image, the second mark position edge image}, {the second mark position edge image, the third mark position edge image, the fourth mark position edge image, the first mark position edge image}.

[0130] The set of edge images with marked positions is input into the color convolutional network to detect the positions of similar patterns and obtain the position color features.

[0131] The position color feature includes the position feature and the color change feature.

[0132] In this embodiment, the color convolution network is a temporal convolutional network (TCN), and the time sequence of the temporal convolutional network is converted into the rotation sequence of the angular position of the corresponding camera device, and the rotation sequence is sequentially input into the temporal convolutional network. The first mark position edge image, the second mark position edge image, the third mark position edge image, and the fourth mark position edge image are sequentially input into the temporal convolutional network in the order of arrangement.

[0133] Multiple position color features are obtained corresponding to multiple marked position edge image sets.

[0134] The position color features are input into the position detection network to detect the position of the special object, and a special segmentation boundary area image containing multiple special segmentation boundary areas is obtained.

[0135] Wherein, the position detection network is a convolutional neural network (CNN).

[0136] Determine whether the colors are connected and obtain the special segmentation boundary area.

[0137] Through the above method, the different positions of an object are judged according to the different depths of the object in the cargo box, and the method of color connectivity during the rotation of an object is used to identify whether the special object is in the area where the object is located.

[0138] Optionally, the detecting the volume of the coal in the cargo box based on the plurality of detected special images to obtain the volume of the coal includes:

[0139] The values ​​of the areas where the special objects exist in the plurality of detected special images are set to 1, so as to obtain a detected special three-dimensional matrix;

[0140] Obtaining a detection volume; the detection volume represents the volume of the area corresponding to the entire image of the detected special three-dimensional image in the cargo box;

[0141] The detected special three-dimensional matrix and the corresponding detection volume are input into a three-dimensional detection network to detect the volume of coal in the cargo box to obtain the volume of coal.

[0142] The three-dimensional detection network finds the volume of the stacked coal by the number of three-dimensional detections 1 and the volume of the area in real life corresponding to the entire image.

[0143] Wherein, the three-dimensional detection network is a three-dimensional convolutional neural network (3D Convolutional Neural Networks, 3DCNN).

[0144] Optionally, the laser three-dimensional data is acquired by a laser radar distributed on the top of the cargo compartment.

[0145] Optionally, the stable cargo image set and the sudden cargo image set are real-time images of multiple objects inside the cargo compartment taken by camera devices distributed around the cargo compartment.

[0146] Example 2

[0147] Based on the above-mentioned laser radar-based train cargo box measurement method, an embodiment of the present invention further provides a laser radar-based train cargo box measurement system, which includes an acquisition module, a special object detection module and a volume detection module.

[0148] The acquisition module is used to obtain laser three-dimensional data, a stable cargo image set and a sudden cargo image set at multiple time points; the laser three-dimensional data includes the distance from each position of the stacked coal in the cargo box to the laser device; the stable cargo image set includes multiple stable cargo images; the multiple stable cargo images represent images containing stacked coal taken at multiple angles when the train cargo box is at a constant speed; the sudden cargo image set includes multiple sudden cargo images; the multiple sudden cargo images represent images containing stacked coal taken at multiple angles after the train cargo box changes speed.

[0149] The common object detection module is used to find partial object areas and obtain multiple detection object positions based on the stable cargo image set and the mutant cargo image set through the target detection network.

[0150] The special object detection module is used to obtain multiple special segmentation boundary area images based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data according to the position and color of the coal stack; the special segmentation boundary area image contains multiple special segmentation boundary areas; the t*m special segmentation boundary area images corresponding to the stable cargo image set at t time points are used as stable special boundary area images; the t*n special segmentation boundary area images corresponding to the sudden cargo image set at t time points are used as motion special boundary area images; based on the stable special boundary area images, the backbone position of the coal stack is found to obtain the object backbone image; multiple stable special boundary area images correspond to multiple object backbone images; based on the motion special image and the corresponding multiple object backbone images, different objects are judged according to different inclination angles during the motion process to obtain detected special images; the detected special images contain multiple positions of stacked coal that are difficult to detect by target detection; multiple detected special images are obtained corresponding to multiple object backbone images.

[0151] The volume detection module is used to detect the volume of coal in the cargo box based on multiple detected special images to obtain the volume of coal.

[0152] Example 3

[0153] The present invention provides a method for measuring a train cargo box based on laser radar, which mainly combines image processing technology and machine learning algorithms to accurately monitor the loading status and safety of the cargo in the cargo box. The following are the detailed steps and workflow of the present invention:

[0154] Step 1: Deploy monitoring equipment, including installing lidar and high-definition cameras at key locations in train cargo compartments.

[0155] Step 2: Collect initial data. Use LiDAR to obtain the three-dimensional volume data of the cargo in the cargo compartment, while the camera captures real-time images of the cargo.

[0156] Step 3: Data preprocessing, formatting the collected images and volume data in preparation for subsequent analysis.

[0157] Step 4: Integrate the analysis results, combining cargo volume data with image analysis results to provide a more comprehensive monitoring report.

[0158] Step 5: Generate a monitoring report and send it to the control center or the mobile devices of relevant personnel in real time to ensure real-time response.

[0159] Example 4

[0160] The present invention provides an efficient laser radar-based train cargo box measurement system, which realizes real-time and automatic monitoring of the cargo status inside the train cargo box by combining high-precision laser radar, high-definition camera and advanced image processing algorithm. The following are the main components of the present invention and its working principle:

[0161] 1. System components: LiDAR: installed on the top of the cargo compartment to measure the volume of the cargo and monitor the loading status of the cargo. HD cameras: distributed around the cargo compartment to capture real-time images of the cargo and the interior of the compartment. Central processing unit: receives data from the LiDAR and camera, and performs data analysis and processing. Data transmission system: responsible for sending the processed information to the control center or mobile device.

[0162] Among them, the working principle and process:

[0163] When the train is running, the lidar continuously scans the interior of the cargo compartment and calculates the volume of the cargo by the time difference of the reflected waves. At the same time, the camera captures video images of the interior of the cargo compartment.

[0164] All data is transmitted in real time to a central processing unit, which uses deep learning algorithms to analyze images, identify cargo types and monitor loading status.

[0165] The central processing unit sends the analysis results to the control center through the data transmission system to achieve real-time monitoring and management of the cargo status.

Claims

1. A method for measuring a train cargo box based on laser radar, characterized in that: include: Obtain laser three-dimensional data, a stable cargo image set and a sudden cargo image set at multiple time points; The laser three-dimensional data includes the distance from each position of the stacked coal in the cargo box to the laser device; the stable cargo image set includes a plurality of stable cargo images; the plurality of stable cargo images represent images containing stacked coal taken at multiple angles when the train cargo box is at a constant speed; the sudden change cargo image set includes a plurality of sudden change cargo images; the plurality of sudden change cargo images represent images containing stacked coal taken at multiple angles after the train cargo box changes speed; Based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, a plurality of special segmentation boundary area images are obtained according to the position and color of the coal stack; the special segmentation boundary area images include a plurality of special segmentation boundary areas; The t*m special segmentation boundary region images corresponding to the stable cargo image set at t time points are used as stable special boundary region images; the t*n special segmentation boundary region images corresponding to the sudden cargo image set at t time points are used as motion special boundary region images; Based on the stable special boundary region image, the backbone position of the coal stack is found to obtain the backbone image of the object; Multiple object backbone images are obtained corresponding to multiple stable special boundary area images; Based on the motion special image and the corresponding multiple object backbone images, a detected special image is obtained by judging according to the different tilt angles of different objects in the motion process; the detected special image includes the positions of stacked coal that are difficult to detect by multiple target detections; and multiple detected special images are obtained corresponding to the multiple object backbone images; Based on multiple detected special images, the volume of the coal in the cargo box is detected to obtain the volume of the coal.

2. The laser radar-based train cargo box measurement method according to claim 1, characterized in that: Based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, a plurality of special segmentation boundary area images are obtained according to the position and color of the coal stack, including: Performing edge detection on the mutant cargo image in the mutant cargo image set or the stable cargo image in the stable cargo image set to obtain an edge image; the edge image includes a plurality of edge closed regions; Obtain n+m edge images corresponding to the n mutant cargo images in the mutant cargo image set and the m stable cargo images in the stable cargo image set; According to the laser three-dimensional data, the distance of the position is determined to obtain a depth distance image; the depth distance image includes a plurality of position areas with different depths; the depth distance image represents the distance from each position in the carriage to the laser device; Segmenting the depth range image according to the positions where the distance mutation occurs to obtain a plurality of position areas; Based on the depth range image including the multiple position areas and the multiple edge images, position and color matching is performed to obtain multiple special segmentation boundary area images.

3. The laser radar-based train cargo box measurement method according to claim 1, characterized in that: The method of finding the backbone position of the coal stack based on the stable special boundary region image and obtaining the backbone image of the object includes: Based on the stable special boundary region image, a stable center point set is obtained; the stable center point set includes multiple stable center points; multiple special segmentation boundary regions of the stable special boundary region image correspond to multiple stable center point sets; Fitting the plurality of stable center points into highly parallel straight lines that are perpendicular to the stable special boundary region image to obtain a backbone straight line of the object; A plurality of object backbone straight lines are obtained corresponding to a plurality of stable special boundary regions of the stable special boundary region image; The plurality of object backbone straight lines are marked in the stable special boundary region image to obtain an object backbone image.

4. The laser radar-based train cargo box measurement method according to claim 1, characterized in that: The method of obtaining the detected special image based on the motion special image and the corresponding multiple object backbone images and judging according to different tilt angles of different objects in the motion process includes: Tracking multiple stable center points corresponding to the object backbone straight line in the object backbone image, finding the moving position of the object backbone straight line in the motion special boundary area image according to the change of the object shape, and obtaining a tilted backbone position set; the tilted backbone position set includes multiple tilted backbone positions; the object backbone image and the motion special boundary area image are images taken by the same camera device at different time points; Obtaining a plurality of inclined backbone position sets corresponding to a plurality of object backbone straight lines in the object backbone image; Fitting a plurality of inclined backbone position sets into an inclined backbone straight line; The angle between the inclined backbone straight line and the edge corresponding to the height of the object backbone image is taken as the inclination angle; If the edge distance between two special segmentation boundary areas in the object skeleton image is less than the edge distance threshold and the inclination angles are equal, the two special segmentation boundary areas are marked as a whole object to obtain multiple special detection object positions; The positions of multiple special detection objects are marked in the object backbone image to obtain a detected special image.

5. The laser radar-based train cargo box measurement method according to claim 3, characterized in that: The step of obtaining a stable center point set based on the stable special boundary region image comprises: A coordinate axis is established according to the stable special boundary region image; the width of the stable special boundary region image corresponds to the abscissa; the height of the stable special boundary region image corresponds to the ordinate; and the lower left corner of the stable special boundary region image corresponds to the coordinate origin; Acquire a range of a vertical coordinate of a smooth special boundary region corresponding to the smooth special boundary region image as a smooth special boundary height range; Add two positions with the same ordinate in the stable special boundary region in the stable special boundary height range to the same height boundary position set; obtain multiple same height boundary position sets corresponding to multiple ordinates; The horizontal coordinates corresponding to the two positions in the same-height boundary position set are used as the horizontal coordinates of the same-height edge; The horizontal coordinates of two edges of the same height in the set of edge positions of the same height are averaged to obtain a stable center point; the vertical coordinate of the stable center point is the vertical coordinate of the edge position of the same height corresponding to the horizontal coordinate of the edge of the same height; Multiple stable center points are obtained corresponding to multiple sets of boundary positions at the same height; The multiple stable center points are added to a stable center point set.

6. The laser radar-based train cargo box measurement method according to claim 2, characterized in that: The method of matching the position and color of the depth distance image containing the multiple position areas with the multiple edge images to obtain multiple special segmentation boundary area images includes: Obtaining a camera angle; the camera angle represents an angle between a depth range image and an edge image; According to the camera angle, the position of the position area corresponding to the depth range image on the edge image is marked by a first convolutional network to obtain a marked position edge image; A plurality of edge images corresponding to a plurality of marked position edge images are obtained; According to the adjacent states of the camera devices corresponding to the marked position edge images, the marked position edge images at multiple angles are arranged in sequence to obtain a marked position edge image set; The labels of the values ​​in the marked position edge image set are shifted backward by 1 bit each time to obtain multiple marked position edge image sets; Input the set of edge images at the marked positions into the color convolutional network to detect the positions of similar patterns and obtain the position color features; Multiple position color features are obtained corresponding to multiple marked position edge image sets; The position color features are input into the position detection network to detect the position of the special object, and a special segmentation boundary area image containing multiple special segmentation boundary areas is obtained; multiple position color features correspond to multiple special segmentation boundary area images.

7. The laser radar-based train cargo box measurement method according to claim 1, characterized in that: The method of detecting the volume of coal in the cargo box based on the plurality of detected special images to obtain the volume of coal includes: The values ​​of the areas where the special objects exist in the plurality of detected special images are set to 1, so as to obtain a detected special three-dimensional matrix; Obtaining a detection volume; the detection volume represents the volume of the area corresponding to the entire image of the detected special three-dimensional image in the cargo box; The detected special three-dimensional matrix and the corresponding detection volume are input into the three-dimensional detection network to detect the volume of coal in the cargo box to obtain the volume of coal.

8. The laser radar-based train cargo box measurement method according to claim 1, characterized in that: The laser three-dimensional data is obtained by a laser radar distributed on the top of the cargo compartment.

9. The laser radar-based train cargo box measurement method according to claim 1, characterized in that: The stable cargo image set and the sudden cargo image set are real-time images of multiple objects inside the cargo compartment taken by camera devices distributed around the cargo compartment.

10. A laser radar-based train cargo box measurement system, characterized in that: include: Acquisition module: obtains laser three-dimensional data, a stable cargo image set and a sudden change cargo image set at multiple time points; the laser three-dimensional data includes the distance from each position of the stacked coal in the cargo box to the laser device; the stable cargo image set includes multiple stable cargo images; the multiple stable cargo images represent images containing stacked coal taken at multiple angles when the train cargo box is at a constant speed; the sudden change cargo image set includes multiple sudden change cargo images; the multiple sudden change cargo images represent images containing stacked coal taken at multiple angles after the train cargo box changes speed; Special object detection module: Based on the sudden cargo image set, the stable cargo image set and the laser three-dimensional data, a plurality of special segmentation boundary area images are obtained according to the position and color of the coal stack; the special segmentation boundary area image contains a plurality of special segmentation boundary areas; the t*m special segmentation boundary area images corresponding to the stable cargo image set at t time points are used as stable special boundary area images; the t*n special segmentation boundary area images corresponding to the sudden cargo image set at t time points are used as motion special boundary area images; based on the stable special boundary area images, the backbone position of the coal stack is found to obtain the object backbone image; multiple stable special boundary area images correspond to multiple object backbone images; Based on the motion special image and the corresponding multiple object backbone images, a detected special image is obtained by judging according to the different tilt angles of different objects in the motion process; the detected special image includes the positions of stacked coal that are difficult to detect by multiple target detections; and multiple detected special images are obtained corresponding to the multiple object backbone images; Volume detection module: Based on multiple detected special images, the volume of coal in the cargo box is detected to obtain the volume of coal.

Images