Bridge collision prediction method and related device based on millimeter wave radar

Abstract

The embodiments of the present application relate to the fields of target detection and millimeter-wave radar, and provide a bridge collision prediction method based on millimeter-wave radar and related devices, the method comprising: obtaining first point cloud data when the millimeter-wave radar in the target ship detects the target bridge, and obtaining a first image obtained by an image acquisition device in the target ship to acquire an image of the target bridge; determining the pier coordinates of the target bridge using the target coordinates of the second point cloud data obtained after filtering the first point cloud data and the first image; predicting the bridge collision according to the pier coordinates, the speed of the target ship, and water flow information to obtain a collision prediction result, which can accurately acquire the pier coordinates of the target bridge according to the point cloud data of the millimeter-wave radar and the image acquired by the image acquisition device, thereby improving the accuracy of the final collision prediction.

Description

Technical Field

[0001] The present application relates to the field of target detection and millimeter wave radar technology, and specifically to a bridge collision prediction method based on millimeter wave radar and related devices. Background Art

[0002] With the development of science and technology, unmanned ships have also developed rapidly. At present, unmanned ships can pass through rivers, but because unmanned ships usually encounter the situation of passing through bridges when passing, it is necessary to accurately locate the position of the bridge piers at this time to predict the collision, and ultimately avoid the unmanned ship from colliding with the bridge piers, thereby causing a shipping accident. In the existing scheme, when accurately locating the position of the bridge piers, radar is usually used for positioning processing, but because the accuracy of the radar in the vertical axis direction is not high, the accuracy of the positioning height is reduced during positioning, resulting in low accuracy in the final collision prediction. Summary of the invention

[0003] The embodiment of the present application provides a bridge collision prediction method and related devices based on millimeter-wave radar, which can accurately obtain the coordinates of the piers of the target bridge according to the point cloud data of the millimeter-wave radar and the images obtained by the image acquisition device, thereby improving the accuracy of the final collision prediction.

[0004] A first aspect of an embodiment of the present application provides a bridge collision prediction method based on millimeter wave radar, the method comprising:

[0005] Acquire first point cloud data when a millimeter wave radar in the target ship detects a target bridge, and acquire a first image obtained by an image acquisition device in the target ship acquiring an image of the target bridge;

[0006] Using the first image to filter the first point cloud data to obtain second point cloud data;

[0007] Acquiring the traveling speed of the target vessel and the jitter information of the target vessel when traveling;

[0008] Correcting the coordinates of the second point cloud data according to the driving speed and the jitter information to obtain target coordinates of the second point cloud data;

[0009] Determine the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image;

[0010] Bridge collision prediction is performed according to the bridge pier coordinates, the travel speed of the target vessel, and water flow information to obtain a collision prediction result.

[0011] In this example, the first point cloud data when the millimeter-wave radar in the target ship detects the target bridge and the first image obtained by the image acquisition device in the target ship to acquire the image of the target bridge are obtained, the first image is used to filter the first point cloud data to obtain second point cloud data, the travel speed of the target ship and the jitter information of the target ship when traveling are obtained, the coordinates of the second point cloud data are corrected according to the travel speed and the jitter information to obtain the target coordinates of the second point cloud data, the pier coordinates of the target bridge are determined according to the target coordinates of the second point cloud data and the first image, and the bridge collision prediction is performed according to the pier coordinates, the travel speed of the target ship, and the water flow information to obtain the collision prediction result. Therefore, the pier coordinates of the target bridge can be accurately acquired according to the point cloud data of the millimeter-wave radar and the image acquired by the image acquisition device, thereby improving the accuracy of the final collision prediction.

[0012] A second aspect of an embodiment of the present application provides a bridge collision prediction device based on millimeter wave radar, the device comprising:

[0013] A first acquisition unit is used to acquire first point cloud data when a millimeter-wave radar in a target ship detects a target bridge, and to acquire a first image obtained by an image acquisition device in the target ship acquiring an image of the target bridge;

[0014] A filtering unit, configured to filter the first point cloud data using the first image to obtain second point cloud data;

[0015] A second acquisition unit, used to acquire the traveling speed of the target vessel and the jitter information of the target vessel when traveling;

[0016] a correction unit, configured to perform correction processing on the coordinates of the second point cloud data according to the driving speed and the jitter information to obtain target coordinates of the second point cloud data;

[0017] a determining unit, configured to determine the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image;

[0018] The prediction unit is used to predict the bridge collision according to the bridge pier coordinates, the speed of the target ship, and the water flow information to obtain the collision prediction result.

[0019] A third aspect of an embodiment of the present application provides a terminal, comprising a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the step instructions in the first aspect of the embodiment of the present application.

[0020] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiments of the present application.

[0021] A fifth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic flow chart of a bridge collision prediction method based on millimeter wave radar is provided for an embodiment of the present application;

[0024] Figure 2 A schematic diagram of the division of bridge piers is provided for an embodiment of the present application;

[0025] Figure 3 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0026] Figure 4 A structural schematic diagram of a bridge collision prediction device based on millimeter-wave radar is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0029] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0030] In order to better understand the bridge collision prediction method based on millimeter wave radar provided in the embodiment of the present application, the bridge collision prediction method in the existing scheme is briefly introduced below. In the existing scheme, the millimeter wave radar and image acquisition device set on the ship are used to obtain point cloud data and image data respectively, and then the coordinates in the point cloud data are converted into coordinates in the image coordinate system, and the bridge contour is used for filtering processing, and finally the coordinates of the bridge in the space coordinate system are calculated using the coordinates of the point cloud data and the coordinates in the image coordinate system, and finally the height of the bridge is determined according to the coordinates in the space coordinate system and subsequent collision prediction is performed, but when performing coordinate correction and conversion processing, the main combined data is the data of the ship itself, and the fixed data set between the millimeter wave radar and the image acquisition device, etc. are converted and processed, but because the ship is inevitably affected by water surface fluctuations, water flow, etc. when traveling in the river, the spatial coordinates of the bridge obtained after the conversion processing based on fixed parameters cannot adapt to the actual river environment, etc., resulting in a decrease in the accuracy of the spatial coordinates of the bridge finally obtained.

[0031] In order to solve the above-mentioned problems, an embodiment of the present application provides a bridge collision prediction method based on millimeter-wave radar, which can accurately extract the pier coordinates of the target bridge to adapt to the river conditions according to the point cloud data of the millimeter-wave radar and the images obtained by the image acquisition device, thereby improving the accuracy of the final collision prediction.

[0032] See also Figure 1 , Figure 1 A flow chart of a bridge collision prediction method based on millimeter wave radar is provided for the embodiment of the present application. Figure 1 As shown, the method includes:

[0033] 101. Acquire first point cloud data when a millimeter wave radar in a target ship detects a target bridge, and acquire a first image obtained by an image acquisition device in the target ship acquiring an image of the target bridge.

[0034] The target vessel may be a vessel operating in a river or the like, and a millimeter wave radar and an image acquisition device may be configured on the target vessel. The image acquisition device may be a monocular camera or a binocular vision camera or the like. The binocular vision camera may obtain the distance between the camera and the photographed object by depth of field calculation or the like. Thus, each pixel point in the first image may be positionally annotated to obtain the initial spatial position coordinates of the pixel point.

[0035] The first point cloud data during millimeter-wave radar detection also has its corresponding spatial position coordinates. The millimeter-wave radar is more accurate than the image acquisition device in ranging and positioning, but the image acquisition device is more accurate than the millimeter-wave radar in locating the contour of the target bridge. Therefore, the data collected by the millimeter-wave radar and the image acquisition device can be combined to perform collision prediction, and the advantages of both can be combined to improve the accuracy of collision prediction. When predicting a bridge collision, it is necessary to predict whether the target ship collides with the pier of the target bridge, and because the pier is connected to the supporting surface of the bridge, it is also necessary to predict whether the target ship collides with the bridge deck under the bridge and the ship when passing. Therefore, in the embodiment of the present application, a custom processing is performed on the pier, such as Figure 2 As shown, the bridge pier includes the pier body and a part of the lower bridge deck. At this time, all bridge collisions are summarized as whether there is a collision with the pier, so as to improve the integrity and continuity of the collision judgment and ultimately improve the accuracy.

[0036] 102. Use the first image to filter the first point cloud data to obtain second point cloud data.

[0037] The method of filtering the first point cloud data using the first image may be: the bridge outline of the target bridge may be extracted according to the first image, and the first point cloud data may be preliminarily filtered according to the spatial coordinates of the pixel points corresponding to the bridge outline to obtain the second point cloud data. The method of preliminary filtering may be: deleting the point cloud data outside the bridge outline envelope to obtain the second point cloud data. The specific deletion method may be to perform a preliminary spatial coordinate conversion according to the bridge outline envelope and convert it into spatial coordinates. The coordinates corresponding to the point cloud data are converted into spatial coordinates, and then the deletion process is performed according to the spatial coordinates. Here, a general coordinate conversion method may be used for conversion processing to obtain spatial coordinates.

[0038] 103. Obtain the traveling speed of the target vessel and the vibration information of the target vessel when traveling.

[0039] Among them, the speed of the target ship can be measured by the speed measuring instrument to obtain the speed value of the speed, and then the driving direction of the target ship is determined as the direction of the speed, and finally the speed is obtained. The jitter information of the target ship when traveling can be understood as the amplitude of the up and down fluctuations of the target ship when traveling. Since the target ship is traveling in the river, etc., it will be affected by the water surface fluctuations, wind direction, etc., so that it will shake when traveling. The jitter here can be understood as the amplitude of the up and down shaking when traveling. When the target ship detects the target bridge while shaking, it will cause deviations in the measurement data. For example, due to the existence of jitter, when the peak and valley values ​​of the jitter are detected, when the target bridge is detected, the corresponding point cloud data will have a certain deviation between the coordinates of the vertical axis, resulting in a decrease in the accuracy of the first point cloud data obtained. At this time, the point cloud data can be corrected according to the jitter information to obtain the corrected data, thereby improving the accuracy of subsequent processing.

[0040] 104. Correct the coordinates of the second point cloud data according to the driving speed and the jitter information to obtain target coordinates of the second point cloud data.

[0041] Among them, the offset information of the target ship on the vertical axis in the coordinate system when traveling can be determined based on the jitter information, and then the coordinates are corrected based on the offset information and the vertical axis correction relationship determined according to the traveling speed to obtain the target coordinates of the second point cloud data. Therefore, the coordinates can be corrected to improve the accuracy of subsequent processing.

[0042] 105. Determine the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image.

[0043] Among them, the bridge contour can be extracted from the first image to obtain the bridge contour information, and the image coordinates of the pixel points can be extracted based on the bridge contour information, and then the image coordinates are converted into space coordinates using the coordinate transfer matrix, and the target coordinate transfer matrix is ​​used to convert the target coordinates into space coordinates, and finally the bridge pier coordinates are determined based on the two space coordinates. Therefore, the final bridge pier coordinates can be determined by combining the contour information in the image and the accurate distance information in the point cloud data, which improves the accuracy of the bridge pier coordinate determination.

[0044] 106. Perform bridge collision prediction based on the bridge pier coordinates, the speed of the target vessel, and water flow information to obtain a collision prediction result.

[0045] Among them, the initial driving trajectory of the target ship can be corrected by using the driving speed and water flow information to obtain the target driving trajectory, and then the target driving trajectory and bridge pier coordinates are used to determine the shortest distance between the target ship and the target bridge pier corresponding to the bridge pier coordinates, and then the water surface fluctuation information is determined according to the water flow information to correct the shortest distance to obtain the collision assessment distance, and finally the collision assessment distance is used to perform collision prediction to obtain the collision prediction result. Therefore, collision prediction can be performed by combining water surface fluctuation information and driving trajectory, etc., which improves the accuracy of collision prediction.

[0046] In this example, the first point cloud data when the millimeter-wave radar in the target ship detects the target bridge and the first image obtained by the image acquisition device in the target ship to acquire the image of the target bridge are obtained, the first image is used to filter the first point cloud data to obtain second point cloud data, the travel speed of the target ship and the jitter information of the target ship when traveling are obtained, the coordinates of the second point cloud data are corrected according to the travel speed and the jitter information to obtain the target coordinates of the second point cloud data, the pier coordinates of the target bridge are determined according to the target coordinates of the second point cloud data and the first image, and the bridge collision prediction is performed according to the pier coordinates, the travel speed of the target ship, and the water flow information to obtain the collision prediction result. Therefore, the pier coordinates of the target bridge can be accurately acquired according to the point cloud data of the millimeter-wave radar and the image acquired by the image acquisition device, thereby improving the accuracy of the final collision prediction.

[0047] In a possible implementation, the travel speed includes the radial travel speed between the target ship and the target bridge and the actual travel speed of the target ship. A method for correcting the coordinates of the second point cloud data according to the travel speed and the jitter information to obtain the target coordinates of the second point cloud data includes:

[0048] A1. Determine, according to the jitter information, the offset information of the target vessel on the vertical axis in the coordinate system when the target vessel is traveling;

[0049] A2, determining a vertical axis correction relational expression of coordinates according to the radial travel speed and the actual travel speed;

[0050] A3. Correct the vertical coordinates in the coordinates of the second point cloud data according to the vertical axis correction relationship and the offset information to obtain the target coordinates of the second point cloud data.

[0051] The jitter curve of the target ship when traveling is determined based on the jitter information. The horizontal axis of the jitter curve is the time axis, and the vertical axis is the offset. A positive offset (positive offset) can indicate that the target ship is offset upwards with a static horizontal plane as a reference; a negative offset (negative offset) can indicate that the target ship is offset downwards with a static horizontal plane as a reference. Thus, the offset mean of the positive offset and the offset mean of the negative offset can be determined based on the jitter curve. The offset mean of the positive offset and the offset mean of the negative offset can be determined as the offset information on the vertical axis.

[0052] Alternatively, the positive offset can be clustered to obtain multiple positive cluster centers, which are the centers where the offset distribution is most concentrated. Therefore, the offset during the data collection period during specific driving can be characterized. The same clustering method is used for the negative offset to obtain multiple negative cluster centers. The offset corresponding to the cluster center can be determined as the offset information on the vertical axis.

[0053] The vertical axis correction relationship can be expressed as follows:

[0054]

[0055] is the radial velocity, is the actual driving speed value, ( ) is the coordinate value of the kth point cloud data, and Z is the vertical axis value after preliminary correction.

[0056] A method for correcting the vertical coordinates in the coordinates of the second point cloud data according to the vertical axis correction relationship and the offset information to obtain the target coordinates of the second point cloud data can be represented by the following formula:

[0057] ,

[0058] ，

[0059] Among them, (t1, t2) is the positive offset time period, is the negative offset time period, is the correction factor, is the variance of the offset, is the mean offset of the positive offset, is the mean offset of the negative offset, is the vertical coordinate of the target coordinates of the second point cloud data.

[0060] In this example, the vertical coordinate in the coordinates of the second point cloud data is corrected through the vertical axis correction relationship and the offset information. Specifically, the final vertical coordinate is determined by the mean value and the variation amplitude in the offset information. Thus, the current fluctuation situation can be reflected in the vertical coordinate, and the vertical coordinate can be obtained dynamically, thereby improving the accuracy of determining the vertical coordinate.

[0061] In a possible implementation, a method for predicting a bridge collision based on the bridge pier coordinates, the speed of the target vessel, and water flow information to obtain a collision prediction result includes:

[0062] B1. Correcting the initial driving track of the target vessel according to the driving speed and the water flow information to obtain a target driving track;

[0063] B2. determining the shortest distance between the target vessel and the target pier corresponding to the pier coordinates according to the target driving track and the pier coordinates;

[0064] B3, determining water surface fluctuation information in a preset water area at the target bridge pier according to the water flow information;

[0065] B4. Correcting the shortest distance according to the water surface fluctuation information to obtain an impact assessment distance;

[0066] B5. Perform collision prediction according to the collision assessment distance to obtain a collision prediction result.

[0067] The travel speed includes the radial travel speed between the target ship and the target bridge and the actual travel speed of the target ship. Therefore, the initial travel trajectory can be corrected according to the actual travel speed of the target ship and the water flow information to obtain the target travel trajectory. Specifically, the actual travel speed value and travel direction of the target ship on the water surface can be determined according to the actual travel speed and the water flow information. For example, the final travel speed value and travel direction can be determined according to the speed synthesis method.

[0068] The initial driving trajectory can be understood as a pre-set driving trajectory for the target vessel when it is traveling in a river, etc. Since the target vessel is actually traveling, its driving trajectory may deviate from the initial driving trajectory due to various factors. Therefore, the final driving speed value and driving direction determined according to the actual driving speed and water flow information can be used to correct the initial driving trajectory to obtain the target driving trajectory.

[0069] The specific correction method can be to obtain the position according to the initial starting point on the initial driving trajectory in combination with the final driving speed value and the driving direction (the position here can be the position obtained by predicting the position based on the driving speed and the driving direction), and finally connect the driving positions to obtain a reference driving trajectory, and fuse the reference driving trajectory with the initial driving trajectory to obtain the target driving trajectory. Since the reference driving trajectory is a calculated trajectory, it may also have a certain deviation from the actual trajectory. Therefore, the target driving trajectory is obtained by fusing the reference driving trajectory with the initial driving trajectory, thereby improving the accuracy of the target driving trajectory acquisition, and at the same time correcting the initial driving trajectory. The trajectory fusion method can be fused in a general trajectory fusion method, which will not be repeated here.

[0070] After obtaining the target driving track, the shortest distance between the target ship and the target pier corresponding to the pier coordinates can be determined according to the spatial position relationship between the track and the pier coordinates. The shortest distance can be understood as the distance between the target ship and the pier based on the longest point of the hull of the target ship.

[0071] The water flow information includes water surface fluctuation information in a preset water area at the target bridge pier, so that the water surface fluctuation information can be extracted from the water flow information. The preset water area can be the water area between the piers when the target ship passes under the target bridge, and the water area can be a circular water area centered on the pier, etc.

[0072] When the target ship passes under the target bridge, it will inevitably be affected by the water flow in the water area between the bridge piers. The fluctuation of the water flow will cause the hull of the target ship to shake (that is, when shaking occurs, the hull will tilt). At this time, the shaking will cause the shortest distance between the target ship and the target bridge pier to change. For example, the shortest distance may increase or decrease. Therefore, the shortest distance can be corrected according to the water surface fluctuation information to obtain the impact assessment distance. Specifically, it can be: the maximum fluctuation distance of the target ship in the shortest distance direction when passing can be determined according to the water surface fluctuation information. The maximum fluctuation distance can be understood as the situation that the hull of the target ship will deviate left and right due to the water surface fluctuation. Therefore, the offset of the target ship hull when it deviates can be predicted according to the water surface fluctuation to obtain the offset range.

[0073] The maximum offset can be to increase the shortest distance (i.e., the obtained collision assessment distance increases), or to reduce the shortest distance (i.e., the obtained collision assessment distance decreases). When the shortest distance increases, the safety of the target ship during passage can be improved. When the shortest distance decreases, if the shortest distance tends to zero after the decrease, or is a negative number, it can indicate that the target ship may collide or will definitely collide. Therefore, the collision prediction can be made according to the final value of the collision assessment distance. Specifically, if the collision assessment distance is larger, the risk of collision is smaller. After exceeding a certain threshold, no collision will occur. If the collision assessment distance is smaller, the risk of collision is higher. After the collision assessment distance is less than zero, it can be determined that it usually occurs. After the collision assessment distance is less than zero, the severity of the collision accident can be determined according to the size of its absolute value. The larger the absolute value, the greater the severity of the accident and the greater the harm; the smaller the absolute value, the smaller the severity of the accident and the smaller the harm.

[0074] In this example, the initial driving trajectory of the target vessel is corrected by using the driving speed and water flow information to obtain the target driving trajectory. The shortest distance between the target vessel and the target pier corresponding to the pier coordinates is determined based on the target driving trajectory and the pier coordinates. Finally, the shortest distance is corrected by using the water surface fluctuation information to obtain the collision assessment distance. The collision prediction is performed based on the collision assessment distance to obtain the collision prediction result, thereby improving the accuracy of the collision prediction.

[0075] In a possible implementation, a method for correcting the shortest distance according to the water surface fluctuation information to obtain an impact assessment distance includes:

[0076] C1. Determining the hull inclination information of the target vessel when it reaches the target bridge pier according to the water surface fluctuation information;

[0077] C2. determining the horizontal coverage distance of the hull according to the hull tilt information;

[0078] C3. Correct the shortest distance according to the horizontal coverage distance to obtain a collision assessment distance.

[0079] Among them, a hull floating model can be constructed according to the water surface fluctuation information and the target ship, and the hull tilt information can be determined according to the hull floating model, and the tilt information includes the maximum tilt. After obtaining the maximum tilt, the horizontal component of the hull when it is tilted in the horizontal plane can be calculated according to the vertical height of the hull and the maximum tilt, and the horizontal component is determined as the horizontal coverage distance (maximum offset distance).

[0080] The method of constructing a hull floating model based on water surface fluctuation information and target ship can be: constructing a hull floating model based on water surface fluctuation information, target ship's load, ship's draft and other historical data, specifically, the hull floating model can be constructed by training a neural network model, etc. It can also be that a scene simulation is performed through a three-dimensional model of the target ship, water surface fluctuation information, target ship's load, target ship's draft, etc., and the simulated data is used to construct a model to obtain a hull floating model.

[0081] After determining the horizontal coverage distance, the horizontal coverage distance can be assigned positive or negative values ​​according to the direction of inclination. For example, if the inclination direction is away from the bridge pier, it can be assigned a positive number, and if the inclination direction is close to the bridge pier, it can be assigned a negative number.

[0082] The horizontal coverage distance and the shortest distance may be summed to obtain the impact assessment distance.

[0083] In a possible implementation, a method for determining the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image includes:

[0084] D1. Extracting the pier contour from the first image to obtain pier contour information;

[0085] D2. Obtaining the image coordinates of the pixel points indicated by the pier contour information to obtain a first image coordinate set;

[0086] D3. According to the coordinate transfer matrix, convert the first image coordinates in the first image coordinate set into pier space coordinates to obtain a first reference pier space coordinate set;

[0087] D4, converting the target coordinates of the second point cloud data into pier space coordinates to obtain a second target pier space coordinate set;

[0088] D5. extracting coordinates corresponding to the second point cloud data from the first reference pier spatial coordinate set to obtain a second reference pier spatial coordinate set;

[0089] D6. Determine the pier coordinates of the target bridge according to the second target pier spatial coordinate set and the second reference pier spatial coordinate set.

[0090] Among them, a general contour extraction method can be used to extract the contour of the first image to obtain the pier contour information. When performing image acquisition, since the image acquisition device is an image acquisition device that can perform position annotation, the image coordinates of the pixel points indicated by the contour information can be extracted from the position information annotated by the image acquisition device to obtain the first image coordinate set.

[0091] The coordinate transfer matrix may be a reference coordinate transfer matrix determined by the relative position relationship between the image acquisition device and the millimeter wave radar; then, according to the relationship between the reference coordinate system used by the millimeter wave radar and the preset spatial coordinate system, the reference coordinate matrix is ​​adjusted to obtain the coordinate transfer matrix. Therefore, the image coordinates can be directly converted into spatial coordinates in the preset spatial coordinate system through the coordinate transfer matrix. The preset spatial coordinate system may be a spatial coordinate system established based on the bridge pier.

[0092] The method of converting the target coordinates of the second point cloud data into the pier space coordinates may specifically refer to the method of converting the first image coordinates into the pier space coordinates, which will not be described in detail here.

[0093] Since the coordinates in the first reference pier spatial coordinate set and the coordinates in the second target pier spatial coordinate set are both spatial coordinate systems established based on the pier, corresponding processing can be performed based on the similarity between the coordinates to obtain the second reference pier spatial coordinate set. The coordinate value in the pier coordinates of the target bridge is determined based on the mean value of the coordinates in the second target pier spatial coordinate set and the second reference pier spatial coordinate set.

[0094] In this example, a converted coordinate set is obtained by performing coordinate conversion processing, and finally the converted coordinate set is used to determine the pier coordinates of the target bridge, thereby improving the accuracy of determining the pier coordinates.

[0095] For the above embodiments, please refer to Figure 3 , Figure 3A schematic diagram of the structure of a terminal provided in an embodiment of the present application, as shown in the figure, includes a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and the program includes instructions for executing the following steps;

[0096] Acquire first point cloud data when a millimeter wave radar in the target ship detects a target bridge, and acquire a first image obtained by an image acquisition device in the target ship acquiring an image of the target bridge;

[0097] Using the first image to filter the first point cloud data to obtain second point cloud data;

[0098] Acquiring the traveling speed of the target vessel and the jitter information of the target vessel when traveling;

[0099] Correcting the coordinates of the second point cloud data according to the driving speed and the jitter information to obtain target coordinates of the second point cloud data;

[0100] Determine the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image;

[0101] Bridge collision prediction is performed according to the bridge pier coordinates, the travel speed of the target vessel, and water flow information to obtain a collision prediction result.

[0102] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that in order to realize the above functions, the terminal includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0103] The embodiment of the present application can divide the terminal into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0104] In line with the above, see Figure 4 , Figure 4 The present invention provides a schematic diagram of a bridge collision prediction device based on a millimeter wave radar. Figure 4 As shown, the device comprises:

[0105] The first acquisition unit 401 is used to acquire first point cloud data when the millimeter wave radar in the target ship detects the target bridge, and to acquire a first image obtained by the image acquisition device in the target ship acquiring the image of the target bridge;

[0106] A filtering unit 402, configured to filter the first point cloud data using the first image to obtain second point cloud data;

[0107] The second acquisition unit 403 is used to acquire the travel speed of the target vessel and the vibration information of the target vessel when traveling;

[0108] A correction unit 404 is used to perform correction processing on the coordinates of the second point cloud data according to the driving speed and the jitter information to obtain the target coordinates of the second point cloud data;

[0109] A determination unit 405, configured to determine the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image;

[0110] The prediction unit 406 is used to perform bridge collision prediction based on the bridge pier coordinates, the speed of the target vessel, and water flow information to obtain a collision prediction result.

[0111] In a possible implementation, the travel speed includes the radial travel speed between the target ship and the target bridge and the actual travel speed of the target ship. The correction unit 404 is specifically configured to:

[0112] Determining, according to the jitter information, offset information of the target vessel on the vertical axis in the coordinate system when traveling;

[0113] Determine a vertical axis correction relational expression of coordinates according to the radial travel speed and the actual travel speed;

[0114] The vertical coordinates in the coordinates of the second point cloud data are corrected according to the vertical axis correction relationship and the offset information to obtain the target coordinates of the second point cloud data.

[0115] In a possible implementation, the prediction unit 406 is specifically configured to:

[0116] Correcting the initial travel track of the target vessel according to the travel speed and the water flow information to obtain a target travel track;

[0117] Determine the shortest distance between the target vessel and the target pier corresponding to the pier coordinates according to the target driving trajectory and the pier coordinates;

[0118] Determine water surface fluctuation information in a preset water area at a target bridge pier according to the water flow information;

[0119] Correcting the shortest distance according to the water surface fluctuation information to obtain an impact assessment distance;

[0120] An impact prediction is performed according to the impact assessment distance to obtain an impact prediction result.

[0121] In a possible implementation, in the aspect of correcting the shortest distance according to the water surface fluctuation information to obtain the impact assessment distance, the prediction unit 406 is specifically used to:

[0122] Determining the hull tilt information of the target vessel when it travels to the target bridge pier according to the water surface fluctuation information;

[0123] Determine the horizontal coverage distance of the hull according to the hull tilt information;

[0124] The shortest distance is corrected according to the horizontal coverage distance to obtain a collision assessment distance.

[0125] In a possible implementation, the determining unit 405 is specifically configured to:

[0126] Extracting the pier contour from the first image to obtain pier contour information;

[0127] Acquire the image coordinates of the pixel points indicated by the pier contour information to obtain a first image coordinate set;

[0128] According to the coordinate transfer matrix, the first image coordinates in the first image coordinate set are converted into pier space coordinates to obtain a first reference pier space coordinate set;

[0129] Convert the target coordinates of the second point cloud data into pier space coordinates to obtain a second target pier space coordinate set;

[0130] Extracting coordinates corresponding to the second point cloud data from the first reference pier spatial coordinate set to obtain a second reference pier spatial coordinate set;

[0131] The pier coordinates of the target bridge are determined according to the second target pier spatial coordinate set and the second reference pier spatial coordinate set.

[0132] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the millimeter-wave radar-based bridge collision prediction methods recorded in the above method embodiments.

[0133] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any one of the millimeter-wave radar-based bridge impact prediction methods recorded in the above method embodiments.

[0134] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0135] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0137] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, the functional units in the various embodiments of the application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0139] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk, etc., and other media that can store program codes.

[0140] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0141] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A bridge collision prediction method based on millimeter wave radar, characterized in that: The method comprises: Acquire first point cloud data when a millimeter wave radar in the target ship detects a target bridge, and acquire a first image obtained by an image acquisition device in the target ship acquiring an image of the target bridge; Using the first image to filter the first point cloud data to obtain second point cloud data; Acquiring the traveling speed of the target vessel and the jitter information of the target vessel when traveling; Correcting the coordinates of the second point cloud data according to the driving speed and the jitter information to obtain target coordinates of the second point cloud data; Determine the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image; Predicting a bridge collision based on the bridge pier coordinates, the speed of the target vessel, and water flow information to obtain a collision prediction result; The travel speed includes the radial travel speed between the target ship and the target bridge and the actual travel speed of the target ship. The correction processing of the coordinates of the second point cloud data according to the travel speed and the jitter information to obtain the target coordinates of the second point cloud data includes: Determining, according to the jitter information, offset information of the target vessel on the vertical axis in the coordinate system when traveling; Determine a vertical axis correction relational expression of coordinates according to the radial travel speed and the actual travel speed; Correcting the vertical coordinates of the second point cloud data according to the vertical axis correction relationship and the offset information to obtain target coordinates of the second point cloud data; The bridge collision prediction is performed according to the bridge pier coordinates, the speed of the target vessel, and the water flow information to obtain the collision prediction result, including: Correcting the initial travel track of the target vessel according to the travel speed and the water flow information to obtain a target travel track; Determine the shortest distance between the target vessel and the target pier corresponding to the pier coordinates according to the target driving trajectory and the pier coordinates; Determine water surface fluctuation information in a preset water area at a target bridge pier according to the water flow information; Correcting the shortest distance according to the water surface fluctuation information to obtain an impact assessment distance; Performing collision prediction according to the collision assessment distance to obtain a collision prediction result; The determining the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image comprises: Extracting the pier contour from the first image to obtain pier contour information; Acquire the image coordinates of the pixel points indicated by the pier contour information to obtain a first image coordinate set; According to the coordinate transfer matrix, the first image coordinates in the first image coordinate set are converted into pier space coordinates to obtain a first reference pier space coordinate set; Convert the target coordinates of the second point cloud data into pier space coordinates to obtain a second target pier space coordinate set; Extracting coordinates corresponding to the second point cloud data from the first reference pier spatial coordinate set to obtain a second reference pier spatial coordinate set; The pier coordinates of the target bridge are determined according to the second target pier spatial coordinate set and the second reference pier spatial coordinate set.

2. The bridge collision prediction method based on millimeter wave radar according to claim 1 is characterized in that: The correcting the shortest distance according to the water surface fluctuation information to obtain the impact assessment distance includes: Determining the hull inclination information of the target vessel when it travels to the target bridge pier according to the water surface fluctuation information; Determine the horizontal coverage distance of the hull according to the hull tilt information; The shortest distance is corrected according to the horizontal coverage distance to obtain a collision assessment distance.

3. A bridge collision prediction device based on millimeter wave radar, characterized in that: For executing the method according to claim 1, the device comprises: A first acquisition unit is used to acquire first point cloud data when a millimeter-wave radar in a target ship detects a target bridge, and to acquire a first image obtained by an image acquisition device in the target ship acquiring an image of the target bridge; A filtering unit, configured to filter the first point cloud data using the first image to obtain second point cloud data; A second acquisition unit, used to acquire the traveling speed of the target vessel and the jitter information of the target vessel when traveling; a correction unit, configured to perform correction processing on the coordinates of the second point cloud data according to the driving speed and the jitter information to obtain target coordinates of the second point cloud data; a determining unit, configured to determine the pier coordinates of the target bridge according to the target coordinates of the second point cloud data and the first image; A prediction unit, used to predict the bridge collision according to the bridge pier coordinates, the speed of the target vessel, and the water flow information, and obtain a collision prediction result; The travel speed includes the radial travel speed between the target ship and the target bridge and the actual travel speed of the target ship. The correction unit is specifically used for: Determining, according to the jitter information, offset information of the target vessel on the vertical axis in the coordinate system when traveling; Determine a vertical axis correction relational expression of coordinates according to the radial travel speed and the actual travel speed; Correcting the vertical coordinates of the second point cloud data according to the vertical axis correction relationship and the offset information to obtain target coordinates of the second point cloud data; The prediction unit is specifically used for: Correcting the initial travel track of the target vessel according to the travel speed and the water flow information to obtain a target travel track; Determine the shortest distance between the target vessel and the target pier corresponding to the pier coordinates according to the target driving trajectory and the pier coordinates; Determine water surface fluctuation information in a preset water area at a target bridge pier according to the water flow information; Correcting the shortest distance according to the water surface fluctuation information to obtain an impact assessment distance; Performing collision prediction according to the collision assessment distance to obtain a collision prediction result; Determine the unit specifically for: Extracting the pier contour from the first image to obtain pier contour information; Acquire the image coordinates of the pixel points indicated by the pier contour information to obtain a first image coordinate set; According to the coordinate transfer matrix, the first image coordinates in the first image coordinate set are converted into pier space coordinates to obtain a first reference pier space coordinate set; Convert the target coordinates of the second point cloud data into pier space coordinates to obtain a second target pier space coordinate set; Extracting coordinates corresponding to the second point cloud data from the first reference pier spatial coordinate set to obtain a second reference pier spatial coordinate set; The pier coordinates of the target bridge are determined according to the second target pier spatial coordinate set and the second reference pier spatial coordinate set.

4. A terminal, characterized in that: The invention comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the bridge collision prediction method based on millimeter wave radar as claimed in claim 1 or 2.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the bridge collision prediction method based on millimeter-wave radar as claimed in claim 1 or 2.

Images