Navigation bridge anti-collision detection early warning method and device based on object imaging and medium

Abstract

The invention discloses a navigation bridge anti-collision detection early warning method and device based on object imaging and a medium, and can be applied to the technical field of traffic. According to the method, after the multi-modal data including the radar data and the video data in the preset detection area is acquired, the radar data is converted to obtain the radio frequency image data, then target identification is performed based on the radio frequency image data and the video data to obtain the target identification result, and then multi-level early warning operation is performed according to the target identification result. Therefore, a large number of devices do not need to be arranged to obtain data, the application cost is effectively reduced, and the early warning accuracy and real-time performance are improved.

Description

Technical Field

[0001] This application relates to the field of transportation technologies, and in particular, to an anti-collision detection and warning method, device, and medium for navigable bridges based on object imaging. Background Art

[0002] In related technologies, since the process of major bridge disaster warning needs to combine advanced sensors, monitoring equipment, and data analysis technologies, the existing major bridge disaster warning methods require a large amount of capital and labor costs. Moreover, due to the diverse data sources of the existing major bridge disaster warning methods, phenomena such as false alarms and missed alarms are likely to occur. At the same time, due to problems such as poor real-time performance in the existing major bridge disaster warning methods, sudden events cannot be dealt with.

[0003] In summary, the technical problems existing in related technologies need to be improved. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose an anti-collision detection and warning method, device, and medium for navigable bridges based on object imaging, which can reduce application costs, improve warning accuracy, and real-time performance.

[0005] To achieve the above object, on the one hand, an embodiment of this application proposes an anti-collision detection and warning method for navigable bridges based on object imaging, and the method includes the following steps:

[0006] Obtain multi-modal data of a preset detection area, where the multi-modal data includes radar data and video data;

[0007] Convert the radar data to obtain radio frequency image data;

[0008] Perform target recognition based on the radio frequency image data and the video data to obtain a target recognition result;

[0009] Perform multi-level warning operations according to the target recognition result, and the multi-level warning operations include pre-warning of the navigable bridge area and warning of the navigable bridge deck.

[0010] In some embodiments, the converting the radar data to obtain radio frequency image data includes:

[0011] Perform a fast Fourier transform on the radar data to obtain first spectrum data;

[0012] Perform low-pass filtering on the first spectrum data to obtain second spectrum data;

[0013] Perform a fast Fourier transform on the second spectrum data to obtain the radio frequency image data.

[0014] In some embodiments, the target recognition based on the radio frequency image data and the video data to obtain a target recognition result includes:

[0015] Identifying a target object and first position information of the target object within a preset detection area based on the video data;

[0016] Identifying the distance and angle of the target object within the preset detection area based on the radio frequency image data;

[0017] Fusing the radio frequency image data and the video data according to the first position information, the target object, and the distance and angle of the target object;

[0018] Performing target recognition based on the fusion result to obtain the target recognition result.

[0019] In some embodiments, the multi-level early warning operation according to the target recognition result includes:

[0020] Performing pre-warning for the navigation bridge area according to the navigation area recognition result of the target recognition result;

[0021] Performing warning for the navigation bridge deck according to the bridge recognition result of the target recognition result.

[0022] In some embodiments, the pre-warning for the navigation bridge area according to the navigation area recognition result of the target recognition result includes:

[0023] When it is determined according to the navigation area recognition result that a ship is within a first preset range, controlling a radar module to perform tracking and warning on the navigation ships within the first preset range;

[0024] When it is determined according to the navigation area recognition result that a ship is within a second preset range, controlling the radar module and the camera module to perform tracking and warning on the navigation ships within the first preset range;

[0025] When it is determined according to the navigation area recognition result that a ship is within a third preset range, controlling the camera module to monitor the pier state in real time and performing warning according to the pier state.

[0026] In some embodiments, the controlling the radar module and the camera module to perform tracking and warning on the navigation ships within the first preset range includes:

[0027] Controlling the radar module and the camera module to collect real-time navigation data of the navigation ships within the first preset range;

[0028] Analyzing the real-time navigation speed and real-time navigation direction of the navigation ships within the first preset range according to the real-time navigation data;

[0029] Track and give early warnings to the navigable vessels within the first preset range according to the real-time navigation speed and real-time navigation direction.

[0030] In some embodiments, the giving of early warnings to the navigable bridge deck according to the bridge recognition result of the target recognition result includes:

[0031] Analyze the bridge health status according to the bridge recognition result of the target recognition result, where the bridge health status includes bridge deformation, bridge inclination, and bridge collapse;

[0032] Control the working status of the bridge deck alarm device according to the bridge health status.

[0033] To achieve the above object, another aspect of the embodiments of the present application proposes a navigable bridge anti-collision detection and early warning device based on object imaging, and the device includes:

[0034] A multimodal data perception module, configured to obtain multimodal data of a preset detection area, where the multimodal data includes radar data and video data; convert the radar data to obtain radio frequency image data;

[0035] A target analysis and detection module, configured to perform target recognition based on the radio frequency image data and the video data to obtain a target recognition result;

[0036] A major disaster discovery and early warning module, configured to perform multi-level early warning operations according to the target recognition result, where the multi-level early warning operations include pre-event early warnings in the navigable bridge area and early warnings for the navigable bridge deck.

[0037] To achieve the above object, another aspect of the embodiments of the present application proposes a computer device, including:

[0038] At least one processor;

[0039] At least one memory, configured to store at least one program;

[0040] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0041] To achieve the above object, another aspect of the embodiments of the present application proposes a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0042] The embodiments of the present application at least include the following beneficial effects: The present application provides a method, device, and medium for anti-collision detection and warning of a navigable bridge based on object imaging. After obtaining multi-modal data including radar data and video data within a preset detection area, the radar data is converted to obtain radio frequency image data, and then target recognition is performed based on the radio frequency image data and video data to obtain a target recognition result. Then, multi-level warning operations are performed according to the target recognition result, so that a large number of devices do not need to be set up to obtain data, effectively reducing the application cost and improving the warning accuracy and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of the method for anti-collision detection and warning of a navigable bridge based on object imaging provided by the embodiments of the present application;

[0044] Figure 2 is a processing flowchart of the radar data provided by the embodiments of the present application;

[0045] Figure 3 is a flowchart of the target recognition process provided by the embodiments of the present application;

[0046] Figure 4 is a schematic diagram of the scenario of multi-level warning operations provided by the embodiments of the present application;

[0047] Figure 5 is a data processing flowchart of the "Active Identification and Pre-warning of Ships in Navigable Bridge Areas" module provided by the embodiments of the present application;

[0048] Figure 6 is a schematic diagram of the application scenario of the "Active Identification and Pre-warning of Ships in Navigable Bridge Areas" module provided by the embodiments of the present application;

[0049] Figure 7 is a data processing flowchart of the "Emergency Rescue of Major Disasters of Bridges and Deck Warning Management" module provided by the embodiments of the present application;

[0050] Figure 8 is a schematic diagram of a preset visualization software platform provided by the embodiments of the present application;

[0051] Figure 9 is a schematic diagram of another preset visualization platform provided by the embodiments of the present application

[0052] Figure 10 is a schematic diagram of the structure of the device for anti-collision detection and warning of a navigable bridge based on object imaging provided by the embodiments of the present application;

[0053] Figure 11 is a schematic diagram of the hardware structure of the computer device provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. When the following description relates to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the embodiments of the present application. They are only examples of devices and methods that are consistent with some aspects of the embodiments of the present application.

[0055] It can be understood that the terms "first", "second", etc. used in the present application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information. Similarly, the second information may also be referred to as the first information. Depending on the context, the words "if", "when" as used herein may be interpreted as "when...", "while...", or "in response to determining".

[0056] The terms "at least one", "a plurality", "each", "any one", etc. used in the present application, at least one includes one, two or more than two, a plurality includes two or more than two, each refers to each of the corresponding plurality, and any one refers to any one of the plurality.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0058] In the related art, major disasters on urban roads include bridges, waterlogging, slopes, collapses, etc. In order to improve the early warning ability of major bridge disasters, a perfect monitoring system needs to be established. By analyzing these monitoring data, major road disasters can be predicted or discovered, and early warning information can be released in a timely manner to ensure the personal and property safety of the people and improve the level and efficiency of urban governance. The necessity of major bridge disaster early warning is mainly reflected in the following aspects:

[0059] Ensuring public safety: Major bridge disasters include collisions of ships navigating in the waterway against the bridge, tilting and collapse of the bridge under external forces, etc. These disasters often cause heavy casualties and property losses. Through the early warning system, relevant information can be released to the public in advance, and people can be reminded in a timely manner to take corresponding preventive measures to reduce the losses and injuries caused by the disasters.

[0060] Improve emergency response efficiency: After a major bridge disaster occurs, a timely emergency response is very important. The warning system can quickly send alerts to relevant departments and rescue personnel, enabling them to reach the disaster site in a timely manner to carry out rescue and emergency work, improve the emergency response efficiency, and reduce the expansion and spread of the disaster.

[0061] Reduce traffic congestion: Major bridge disasters often lead to traffic jams, seriously affecting road traffic. Through the warning system, drivers and traffic management departments can be notified in advance to take measures to dredge traffic, reduce traffic congestion, and ensure smooth roads.

[0062] Enhance social sense of security: Major bridge disasters have a serious impact on social order and people's lives, easily triggering public panic and uneasiness. Through the warning system, accurate information can be transmitted to the public, increasing the public's awareness and understanding of disasters, enhancing the social sense of security, and reducing the spread of panic.

[0063] However, since the current major bridge disaster warning process needs to combine advanced sensors, monitoring equipment, and data analysis technologies, the existing major bridge disaster warning methods require a large amount of capital and human costs. Moreover, due to the diverse data sources of the existing major bridge disaster warning methods, phenomena such as false alarms and missed alarms are likely to occur. At the same time, due to problems such as poor real-time performance in the existing major bridge disaster warning methods, sudden events cannot be dealt with.

[0064] In view of this, in the embodiments of the present application, a navigation bridge anti-collision detection and warning method, device, and medium based on object imaging are provided. After obtaining multi-modal data including radar data and video data in a preset detection area, the radar data is converted to obtain radio frequency image data, and then target recognition is performed based on the radio frequency image data and video data to obtain a target recognition result. Then, multi-level warning operations are performed according to the target recognition result, so that a large number of devices do not need to be set up to obtain data, effectively reducing the application cost, and improving the warning accuracy and real-time performance.

[0065] The anti-collision detection and warning method for navigable bridges based on object imaging provided by the embodiments of the present application relates to the field of traffic technologies. The anti-collision detection and warning method for navigable bridges based on object imaging provided by the embodiments of the present application can be applied to a terminal, can also be applied to a server, or can be software running on a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, etc., but is not limited thereto; the server side can be configured as an independent physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the anti-collision detection and warning method for navigable bridges based on object imaging, etc., but is not limited to the above forms.

[0066] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0067] The following specifically elaborates on the embodiments of the present application with reference to the accompanying drawings:

[0068] Figure 1 is an optional flowchart of the anti-collision detection and warning method for navigable bridges based on object imaging provided by the embodiments of the present application, Figure 1 The method in may include but is not limited to steps S110 to S140:

[0069] Step S110, obtain multi-modal data of a preset detection area, where the multi-modal data includes radar data and video data;

[0070] Step S120, convert the radar data to obtain radio frequency image data;

[0071] Step S130: Perform target recognition based on the radio frequency image data and the video data to obtain a target recognition result;

[0072] Step S140: Perform multi-level warning operations according to the target recognition result, where the multi-level warning operations include pre-warning of the navigation bridge area and navigation bridge deck warning.

[0073] It can be understood that the multi-modal data in this embodiment can be collected by a multi-modal collection module. Among them, the multi-modal collection module is mainly responsible for collecting radar wave information and video information of the target to be recognized in the surrounding environment of the bridge, and realizing data compression and wireless transmission. The video perception system is completed by the existing CCTV video monitoring technology. Specifically, the millimeter-wave radar system is a radar system using the millimeter-wave frequency band, mainly used for detecting and measuring the distance, speed and direction of a long-distance target. The working principle of the millimeter-wave radar system is to emit millimeter-wave signals. When the signal encounters a target, part of the signal will be reflected back to the radar receiver, and the receiver will process the received signal to obtain information such as the position, angle and speed of the target.

[0074] It can be understood that the millimeter-wave radar can obtain the distance, speed and direction of the target by measuring the time delay, Doppler frequency shift and signal strength of the reflected signal. Among them,

[0075] The process of obtaining the distance can be that the millimeter-wave radar calculates the distance between the target and the radar by measuring the time delay of the reflected signal. Among them, the millimeter-wave radar can send a short pulse signal and measure the time difference between the transmission and reception of this signal, and then by multiplying the time difference by the speed of light, the distance of the target can be calculated.

[0076] The process of obtaining the speed can be that the millimeter-wave radar can calculate the speed of the target by measuring the Doppler frequency shift of the reflected signal. When the target moves relative to the radar, the frequency of the reflected signal will change. The millimeter-wave radar can measure this frequency change to calculate the speed of the target.

[0077] The process of obtaining the direction can be that the millimeter-wave radar determines the direction of the target by analyzing the phase difference between the transmitted signal and the reflected signal. This can be achieved by using multiple antennas on the radar. By measuring the signals from different directions, the radar can determine the direction of the target.

[0078] In the embodiment of the present application, the radar data obtained in this embodiment can be converted to obtain radio frequency (RF) image data. Specifically, as Figure 2As shown, the RF image uses radar range-azimuth coordinates and can be described as a bird's-eye view (BEV) representation, where the x-axis represents the azimuth angle (in degrees) and the y-axis represents the range (in distance). The FMCW radar transmits continuous chirps and receives the reflected echoes from obstacles. After the echoes are received and preprocessed, in this embodiment, the obtained radar data is subjected to a fast Fourier transform (FFT) to obtain the first spectral data, and then the reflected range can be estimated. Then, a low-pass filter (LPF) is used to remove the high-frequency noise in all chirps of each frame in the first spectral data at a rate of 30 FPS. After the LPF, in this embodiment, a second FFT is performed on the second spectral data along different receiver antennas to estimate the azimuth angle of the reflection and obtain the final RF image. After the radar data is converted into a radio frequency image, it becomes a format similar to an image sequence and is then processed by an image-based CNN.

[0079] In the embodiment of the present application, after obtaining the video data, several frames of images in the video data are obtained, and then the several frames of images are input into a preset feature filter for image extraction to extract the target object of interest. Among them, in this embodiment, a neural network model can be used to perform deep training on the redefined training object to obtain a feature matrix, and then this feature matrix is used as the preset feature filter for the target object of interest. Specifically, through its internal complex calculation mechanism, the feature filter can efficiently filter out the background information irrelevant to the target object of interest, thereby only retaining the target object of interest in the image. This processing process not only greatly reduces the amount of image data transmission, but also significantly improves the purity and utilization rate of the data. After the image data is processed by the feature filter, the amount of data is significantly reduced and no longer contains irrelevant interfering backgrounds, so that the occupancy of network bandwidth by data transmission can be greatly reduced, and the processing burden on computer hardware devices can be effectively reduced, improving the operating efficiency of the overall imaging system.

[0080] It can be understood that in this embodiment, target recognition is performed after obtaining the radio frequency image data and the video data. Specifically, in this embodiment, the target object and the first position information of the target object in the preset detection area can be recognized based on the target image of interest corresponding to the video data, and at the same time, the distance and angle of the target object in the preset detection area can be recognized based on the radio frequency image data. Then, the radio frequency image data and the video data are fused according to the first position information, the target object, and the distance and angle of the target object, and target recognition is performed according to the fusion result to obtain the target recognition result.

[0081] Exemplarily, as Figure 3 shown, the target recognition process of this embodiment includes but is not limited to the following steps:

[0082] Data preprocessing: Preprocess the data obtained by the monocular camera and the millimeter-wave radar, including data denoising, calibration, coordinate system conversion, etc.

[0083] Object detection and tracking: Use a monocular camera to detect and track objects, identify target objects in the environment, and obtain the three-dimensional position information (first position information) of the target objects. At the same time, use millimeter-wave radar to detect and track obstacles and obtain information such as the distance and angle of obstacles.

[0084] Data fusion: The data obtained by the monocular camera and the millimeter-wave radar are fused to generate more accurate and complete environmental perception information. The specific fusion methods include:

[0085] Data alignment: Align the data obtained by the monocular camera and the millimeter-wave radar to ensure that the information obtained by both is in the same coordinate system.

[0086] Data association: The data obtained by the monocular camera and the millimeter-wave radar are associated to match the objects detected by the monocular camera and the obstacles detected by the millimeter-wave radar.

[0087] Data fusion: The three-dimensional object position information obtained by the monocular camera and the obstacle distance, speed and angle information obtained by the millimeter wave radar are integrated to generate a more accurate and complete environmental perception information, and then the final target recognition result can be obtained. The specific fusion method can use Kalman filtering, extended Kalman filtering and other methods.

[0088] It is understandable that after completing the target identification, the early warning terminal equipment can be grouped into extremely simple early warning points according to the actual situation of each preset detection area, so as to issue the corresponding early warning information in a timely and accurate manner. Among them, the early warning information can be in the form of data information or voice information. In an embodiment of the present application, this embodiment can perform advance early warning in the navigable bridge area based on the navigable area identification result of the target identification result, and can also perform navigable bridge deck early warning based on the bridge identification result of the target identification result. Specifically, this embodiment can perform early warning operations by setting the "active identification and advance warning of ships in the navigable bridge area" module and the "bridge major disaster emergency rescue and bridge deck warning management" module respectively.

[0089] Specifically, the execution process of the "Active Identification and Advance Warning of Vessels in Navigable Bridge Areas" module includes but is not limited to the following steps:

[0090] When it is determined that the ship is located in the first preset range according to the navigation area identification result, the radar module is controlled to track and warn the navigation ship in the first preset range;

[0091] When it is determined that the ship is located in the second preset range according to the navigation area identification result, the radar module and the camera module are controlled to track and warn the navigation ship in the first preset range;

[0092] When it is determined that the ship is located within the third preset range according to the recognition result of the navigable area, control the camera module to monitor the pier status in real time and issue a warning according to the pier status.

[0093] It can be understood that the process of controlling the radar module and the camera module to track and warn the navigable ships within the first preset range can be to control the radar module and the camera module to collect the real-time navigation data of the navigable ships within the first preset range, and then analyze the real-time navigation speed and real-time navigation direction of the navigable ships within the first preset range according to the real-time navigation data, and then track and warn the navigable ships within the first preset range according to the real-time navigation speed and real-time navigation direction.

[0094] Exemplarily, as Figure 4 shown and Figure 5 shown, first of all, in the "Active Identification and Pre-Warning of Ships in Navigable Bridge Areas" module, there are three warning mechanisms. The first level (sensing area): when the ship is within the distance of the first preset range (1500 - 500 meters) from the bridge, the intelligent perception and tracking of the ship are completed through the millimeter-wave radar, and the detection and trajectory tracking of the position, course, and speed of the ship on the waterway are completed; the second level (warning area): when the ship is within the distance of the second preset range (500 - 100 meters) from the bridge, the detection and warning of the over-height, yaw, and overspeed states of the navigating ship are completed through the multi-modal data fusion method of the millimeter-wave radar and the dual-wavelength video camera; the third level (monitoring and alarm area): when the ship is within the distance of the third preset range (100 - -100 meters) of the bridge opening in the navigable bridge area, the pier status at both ends of the navigable bridge opening is monitored in real time through the wide-angle camera, the detection and warning of the pier collision status in the navigable bridge area are completed, and according to the risk levels defined by the competent department, it is divided into low, medium, and high risk warnings.

[0095] Taking Figure 6 the scenario shown as an example, the "Active Identification and Pre-Warning of Ships in Navigable Bridge Areas" module of this embodiment can obtain the real-time navigation data of the ship's position, height, direction, speed and other status information through the network high-definition dual-wavelength intelligent camera and the millimeter-wave radar, and identify the ship type. Then, these real-time navigation data are processed by data fusion AI and transmitted to the decision center. In the decision center, the received information is processed and evaluated using the preset anti-collision safety rules, and finally a warning message is issued. The warning message will be conveyed to the user and the superior department through means such as a significant reminder signal or a sound alarm prompt.

[0096] It can be understood that the execution process of the "Emergency Rescue of Major Bridge Disasters and Bridge Deck Alert Management" module includes but is not limited to the following steps:

[0097] Analyze the bridge health status based on the bridge recognition result of the target recognition result, where the bridge health status includes bridge deformation, bridge inclination, and bridge collapse;

[0098] Control the working status of the bridge deck alarm device according to the bridge health status.

[0099] Exemplarily, as Figure 4 shown, in the "Bridge Major Disaster Emergency Rescue and Bridge Deck Alert Management" module, the IVth-level (bridge alert area) early warning mechanism is correspondingly implemented. Through four shore-based dual-wavelength cameras on both sides of the river, real-time all-weather bridge health status information is obtained, and the AI processing algorithm based on object imaging technology is used to complete the detection and early warning of the damaged state of the bridge, and corresponding low, medium, and high-level risk early warnings are issued.

[0100] Specifically, the early warning process of the "Bridge Major Disaster Emergency Rescue and Bridge Deck Alert Management" module can monitor the real-time condition of the bridge through the all-weather video of the shore-based dual-wavelength intelligent camera on the river bank, and use the AI processing algorithm based on object imaging technology to automatically monitor different health conditions of the bridge (such as deformation, inclination, collapse, etc.). Then, through the decision-making of the edge computing center, the alarm information is automatically sent to the drivers and passengers on the bridge deck in a timely manner through the bridge deck LED display, sound and light alarm, etc., and reported to the competent department in a timely manner. It can be understood that the spatial computing algorithm principle based on object imaging technology is as follows: First, the shore-based camera obtains the all-weather image information of the entire bridge area, then the observation targets installed on the bridge are intelligently located through object imaging technology (equidistantly distributed, with a 30-meter interval between two targets), and the key point coordinate values of the targets are obtained. Then, the bridge displacement caused by various external physical events is detected, so as to judge three different alarm levels. Exemplarily, as Figure 7 shown, judge whether the bridge health detection result based on object imaging belongs to low risk. If so, upload the alarm information to the background competent department. Otherwise, continue to judge whether it belongs to medium-risk alarm. If it belongs to medium-risk alarm, an alarm prompt is given to the external devices of the bridge deck lane. Otherwise, continue to judge whether it belongs to high-risk alarm. If so, the bridge deck road is closed. Otherwise, the camera data is obtained again.

[0101] In some embodiments, the method of this embodiment can also be applied to a visualization software platform. As Figure 8As shown in the figure, the visualization software platform includes a platform layer and an application layer. Among them, the platform layer mainly completes data recording and management. The application layer mainly realizes the release and management of visualization information, including functions such as real-time warning, real-time monitoring, real-time positioning, historical data backtracking, weather forecast, and collection of basic ship information. Through these functions of the application layer, users can intuitively understand the real-time conditions of waterways and bridges, obtain warning information in a timely manner, and take corresponding countermeasures. At the same time, the historical data backtracking function can also provide strong support for accident investigation and decision-making analysis.

[0102] In some other embodiments, the multi-level warning operation results obtained in this application and the acquired multi-modal data can both be stored in Figure 9 the database of the preset visualization platform shown in the figure. It can be understood that the preset visualization platform can be a Web visualization platform constructed by combining the Django framework and PyEcharts visualization technology, which is used to provide an efficient, secure, easy-to-maintain Web application development solution with powerful data visualization capabilities. Specifically, as Figure 9 shown in the figure, when a customer needs to query the data in the database, an http request can be sent to the WEB server through the presentation layer in the client to request the database server to call the data in the SQL database, and then the WEB server controls the presentation layer in the client to perform HTML page display according to the result database returned by the database server.

[0103] In summary, through the use of object imaging technology in the embodiments of this application, the types of sensors used in the system can be greatly reduced, and correspondingly, the monitoring devices corresponding to the backend can be greatly reduced. Subsequently, the data analysis technology also greatly reduces the amount of data stored, transmitted, and processed, as well as the system noise and interference introduced by multiple sensors, and can reduce the maintenance cost and improve the real-time performance of early warning.

[0104] Referring to Figure 10 , the embodiments of this application provide a collision prevention and detection warning device for navigable bridges based on object imaging. The device includes:

[0105] A multi-modal data perception module for acquiring multi-modal data of a preset detection area, where the multi-modal data includes radar data and video data; converting the radar data to obtain radio frequency image data;

[0106] A target analysis and detection module for performing target recognition based on the radio frequency image data and video data to obtain a target recognition result;

[0107] A major disaster discovery and warning module for performing multi-level warning operations according to the target recognition result, where the multi-level warning operations include pre-warning of the navigable bridge area and warning of the navigable bridge deck.

[0108] It can be understood that the content in the above method embodiments is applicable to the device embodiments of the present application. The functions specifically implemented by the device embodiments of the present application are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0109] An embodiment of the present application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the above method is implemented. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0110] It can be understood that the content in the above method embodiments is applicable to the device embodiments of the present application. The functions specifically implemented by the device embodiments of the present application are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0111] Please refer to Figure 11 , Figure 11 which schematically shows the hardware structure of a computer device according to another embodiment. The computer device includes:

[0112] A processor 1010, which can be implemented by using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present application;

[0113] A memory 1020, which can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 1020 can store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020, and are called by the processor 1010 to execute the above method of the embodiments of the present application;

[0114] An input / output interface 1030, which is used to implement information input and output;

[0115] A communication interface 1040, which is used to implement communication interaction between this device and other devices, and can implement communication through a wired method (such as USB, network cable, etc.) or through a wireless method (such as a mobile network, WIFI, Bluetooth, etc.);

[0116] A bus 1050 transmits information between various components of the device, such as a processor 1010, a memory 1020, an input / output interface 1030, and a communication interface 1040.

[0117] Among them, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 achieve communication connections with each other inside the device through the bus 1050.

[0118] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above method is implemented.

[0119] It can be understood that the content in the above method embodiments is applicable to the embodiments of this storage medium. The functions specifically implemented by the embodiments of this storage medium are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0120] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0121] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0122] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown, or combine some steps, or different steps.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] Those of ordinary skill in the art will understand that all or some of the steps in the methods disclosed above, and the functional modules / units in systems and devices, can be implemented as software, firmware, hardware, or a suitable combination thereof.

[0125] As used in the specification of this application and the above-mentioned drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data may be interchanged under appropriate circumstances so that the embodiments of this application described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0126] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Here, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or a similar expression means any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0127] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above-mentioned unit division is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be in electrical, mechanical, or other forms.

[0128] The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional unit in various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs and other various media that can store programs.

[0131] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. However, this does not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall fall within the scope of the rights of the embodiments of the present application.

Claims

1. A navigation bridge anti-collision detection and early warning method based on object imaging, characterized in that: The method comprises the following steps: Acquire multimodal data of a preset detection area, wherein the multimodal data packet includes radar data and video data; Converting the radar data to obtain radio frequency image data; Performing target recognition based on the radio frequency image data and the video data to obtain a target recognition result; A multi-level warning operation is performed according to the target identification result, and the multi-level warning operation includes advance warning of the navigation bridge area and navigation bridge deck warning.

2. The method according to claim 1, characterized in that The converting the radar data to obtain radio frequency image data includes: Performing a fast Fourier transform on the radar data to obtain first spectrum data; Performing low-pass filtering on the first spectrum data to obtain second spectrum data; Perform fast Fourier transform on the second spectrum data to obtain the radio frequency image data.

3. The method according to claim 1, characterized in that The performing target recognition based on the radio frequency image data and the video data to obtain a target recognition result includes: Identify a target object within the preset detection area and first position information of the target object based on the video data; Identify the distance and angle of the target object within the preset detection area based on the radio frequency image data; fusing the radio frequency image data and the video data according to the first position information, the distance and angle between the target object and the target object; Target recognition is performed according to the fusion result to obtain the target recognition result.

4. The method according to claim 1, characterized in that: The multi-level warning operation is performed according to the target identification result, including: Providing advance warning of the navigable bridge area based on the navigable area identification result of the target identification result; A navigable bridge deck warning is performed based on the bridge recognition result of the target recognition result.

5. The method according to claim 4, characterized in that The prior warning of the navigable bridge area according to the navigable area identification result of the target identification result includes: When it is determined according to the navigation area identification result that the ship is located in the first preset range, the radar module is controlled to track and warn the navigation ship in the first preset range; When it is determined that the ship is located in the second preset range according to the navigation area identification result, the radar module and the camera module are controlled to track and warn the navigation ship in the first preset range; When it is determined that the ship is located in the third preset range according to the navigation area identification result, the camera module is controlled to monitor the state of the pier in real time and issue an early warning according to the state of the pier.

6. The method according to claim 5, characterized in that The controlling radar module and the camera module to track and warn the navigable ships within the first preset range includes: Controlling the radar module and the camera module to collect real-time navigation data of navigable ships within the first preset range; Analyzing the real-time navigation speed and real-time navigation direction of the navigable ships within the first preset range according to the real-time navigation data; Tracking and warning are carried out for navigable ships within the first preset range according to the real-time navigation speed and the real-time navigation direction.

7. The method according to claim 4, characterized in that The bridge deck warning according to the bridge recognition result of the target recognition result includes: Analyzing the health status of the bridge according to the bridge recognition result of the target recognition result, wherein the health status of the bridge includes bridge deformation, bridge tilt and bridge collapse; The working state of the bridge deck alarm device is controlled according to the health state of the bridge.

8. A navigation bridge anti-collision detection and warning device based on object imaging, characterized in that: The device comprises: A multimodal data sensing module is used to obtain multimodal data of a preset detection area, wherein the multimodal data packet includes radar data and video data; and convert the radar data to obtain radio frequency image data; A target analysis and detection module, used to perform target recognition based on the radio frequency image data and the video data to obtain a target recognition result; The major disaster discovery and warning module is used to perform multi-level warning operations based on the target identification results, and the multi-level warning operations include advance warnings in navigation bridge areas and warnings on navigation bridge decks.

9. A computer device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images