Deep sea operation equipment safety monitoring method and system based on panoramic sonar image

Abstract

The invention provides a safety monitoring method and system for deep-sea operation equipment based on a panoramic sonar image, and the method comprises the steps: obtaining sonar signals returned by all sonar equipment after seabed detection when the deep-sea operation equipment is operated, constructing the panoramic sonar image according to the sonar signals, carrying out the pixel measurement of the panoramic sonar image, and obtaining the safety of the deep-sea operation equipment based on the panoramic sonar image. The method comprises the steps of obtaining coordinate values of a detected target object in an actual space and the size of the target object, determining the current safety level of deep-sea operation equipment according to the coordinate values of the target object in the actual space and the size of the target object, and controlling the deep-sea operation equipment to execute corresponding operation actions according to the current safety level of the deep-sea operation equipment. According to the invention, a safety early warning mechanism for the deep-sea operation equipment is realized, the operation risk of the deep-sea operation equipment is avoided, and the service life of the deep-sea operation equipment is prolonged.

Description

Technical Field

[0001] The present invention relates to the technical field of deep-sea operations, and in particular to a method, device and system for safety monitoring of deep-sea operation equipment based on panoramic sonar images, a terminal device and a computer-readable storage medium. Background Art

[0002] During deep-sea trenching, the monitoring system can monitor the trenching operation. However, due to the complex geological conditions and turbid offshore waters, the existing monitoring system still cannot fully understand the trenching operation environment, which poses a safety hazard and shortens the trenching machine's service life. Summary of the invention

[0003] The main purpose of the present invention is to provide a deep-sea operating equipment safety monitoring method, device and system based on panoramic sonar images, terminal equipment and computer-readable storage medium, aiming to solve the technical problem that the existing monitoring system is still unable to fully understand the working environment of the trencher, there are safety hazards, thereby shortening the service life of the trencher.

[0004] In a first aspect, the present invention provides a method for safety monitoring of deep-sea operation equipment based on panoramic sonar images, wherein sonar equipment is installed in each direction of the deep-sea operation equipment, and the method comprises: Acquiring sonar signals returned by all the sonar devices after seabed detection when the deep-sea operation equipment is in operation; constructing a panoramic sonar image according to the sonar signal; Performing pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target object in the actual space and the size of the target object; Determining the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; According to the current safety level of the deep-sea operation equipment, the deep-sea operation equipment is controlled to perform corresponding operation actions.

[0005] In a specific embodiment, the pixel measurement of the panoramic sonar image to obtain the coordinate value of the detected target object in the actual space and the size of the target object includes: Calibrate the pixel resolution of the panoramic sonar image to obtain a calibration ratio; Measuring the pixel position and the number of pixels occupied by the target object in the panoramic sonar image; The detected coordinate value of the target object in the actual space and the size of the target object are calculated according to the calibration ratio, the pixel position and the pixel quantity.

[0006] In a specific embodiment, calibrating the pixel resolution of the panoramic sonar image to obtain a calibration ratio includes: Acquire the sonar signal of a standard object of known distance and known size within the detection range of the sonar device; constructing a panoramic sonar image for calibration according to the sonar signal; Measuring the pixel position and the number of pixels occupied by the standard object in the panoramic sonar image used for calibration; According to the pixel position and the number of pixels occupied by the standard object in the panoramic sonar image used for calibration, the proportional relationship between the pixels of the panoramic sonar image used for calibration and the distance and the known size is determined to obtain a calibration ratio.

[0007] In a specific embodiment, determining the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object includes: Predicting the operation risk of the deep-sea operation equipment according to the coordinate value of the target object in the actual space, the size of the target object, and the operation status of the deep-sea operation equipment; The current safety level of the deep-sea operation equipment is determined based on the operation risk of the deep-sea operation equipment and a preset safety threshold.

[0008] In a specific embodiment, the safety level includes level one operation risk, level two operation risk and level three operation risk, and the controlling the deep-sea operation equipment to perform corresponding operation actions according to the current safety level of the deep-sea operation equipment includes: When it is determined that the current safety level of the deep-sea operation equipment is the first-level operation risk, controlling the deep-sea operation equipment to continue to perform the current operation; When it is determined that the current safety level of the deep-sea operation equipment is the second-level operation risk, controlling the deep-sea operation equipment to perform an evasive action; When it is determined that the current safety level of the deep-sea operation equipment is the third-level operation risk, the deep-sea operation equipment is controlled to immediately stop operation.

[0009] In a specific embodiment, the method further comprises: Control a display device to display the panoramic sonar image.

[0010] In a second aspect, the present invention provides a deep-sea operation equipment safety monitoring device based on panoramic sonar images, wherein sonar equipment is installed in each direction of the deep-sea operation equipment, and the device comprises: A sonar signal acquisition module, used to acquire the sonar signals returned by all the sonar devices after seabed detection when the deep-sea operation equipment is in operation; A panoramic sonar image construction module, used to construct a panoramic sonar image according to the sonar signal; A pixel measurement module is used to perform pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target in the actual space and the size of the target; A safety level determination module, used to determine the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; The operation action execution module is used to execute corresponding operation actions according to the current safety level of the deep-sea operation equipment.

[0011] In a third aspect, the present invention provides a deep-sea operation equipment safety monitoring system based on panoramic sonar images, comprising a plurality of sonar devices, deep-sea operation equipment, a panoramic sonar image construction device and a control device; A plurality of the sonar devices are respectively installed in various directions of the deep-sea operation equipment, and are used to obtain sonar signals returned after seabed detection when the deep-sea operation equipment is in operation; The panoramic sonar image construction device is electrically connected to the plurality of sonar devices and is used to: construct a panoramic sonar image according to the sonar signal; and perform pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target in the actual space and the size of the target; The control device is electrically connected to the panoramic sonar image construction device and the deep-sea operation equipment, respectively, and is used to: determine the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; and, according to the current safety level of the deep-sea operation equipment, control the deep-sea operation equipment to perform corresponding operating actions.

[0012] In a fourth aspect, the present invention provides a terminal device, comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements a deep-sea operation equipment safety monitoring method based on panoramic sonar images as described in the first aspect.

[0013] In a fifth aspect, the present invention provides a computer-readable storage medium, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a deep-sea operation equipment safety monitoring method based on panoramic sonar images as described in the first aspect.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: by acquiring the sonar signals returned by all sonar devices after seabed detection when the deep-sea operating equipment is in operation, a panoramic sonar image is constructed according to the sonar signals, pixel measurement is performed on the panoramic sonar image, and the coordinate values ​​of the detected target objects in the actual space and the size of the target objects are obtained; according to the coordinate values ​​of the target objects in the actual space and the size of the target objects, the current safety level of the deep-sea operating equipment is determined; according to the current safety level of the deep-sea operating equipment, the deep-sea operating equipment is controlled to perform corresponding operating actions, thereby realizing a safety early warning mechanism for the deep-sea operating equipment, avoiding the operating risks of the deep-sea operating equipment, and increasing the service life of the deep-sea operating equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of a method for safety monitoring of deep-sea operation equipment based on panoramic sonar images provided by one embodiment of the present invention; Figure 2 It is a structural schematic diagram of a deep-sea operation equipment safety monitoring system based on panoramic sonar images provided by an embodiment of the present invention; Figure 3 It is a schematic structural diagram of a deep-sea operation equipment safety monitoring device based on panoramic sonar images provided by an embodiment of the present invention; Figure 4 It is a structural diagram of a terminal device provided by an embodiment of the present invention.

[0016] in: The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0017] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0018] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0019] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0020] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0021] See also Figure 1 , Figure 1 The present invention is a flowchart of a method for safety monitoring of deep-sea operating equipment based on panoramic sonar images provided by an embodiment of the present invention.

[0022] A method for safety monitoring of deep-sea operation equipment based on panoramic sonar images according to an embodiment of the present invention, wherein sonar equipment is installed in each direction of the deep-sea operation equipment, comprises the following steps: S100, obtaining sonar signals returned by all the sonar devices after seabed detection when the deep-sea operation equipment is in operation; S200, constructing a panoramic sonar image according to the sonar signal; S300, performing pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target object in the actual space and the size of the target object; S400, determining the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; S500. Control the deep-sea operation equipment to perform corresponding operation actions according to the current safety level of the deep-sea operation equipment.

[0023] Specifically, the deep-sea operation equipment safety monitoring method based on panoramic sonar images in the embodiment of the present invention can be applied to a deep-sea operation equipment safety monitoring system based on panoramic sonar images. Figure 2 , Figure 2 It is a structural schematic diagram of a deep-sea operation equipment safety monitoring system based on panoramic sonar images provided by an embodiment of the present invention.

[0024] An embodiment of the present invention provides a deep-sea operation equipment safety monitoring system based on panoramic sonar images, comprising a plurality of sonar devices, deep-sea operation equipment, a panoramic sonar image building device and a control device; A plurality of the sonar devices are respectively installed in various directions of the deep-sea operation equipment, and are used to obtain sonar signals returned after seabed detection when the deep-sea operation equipment is in operation; The panoramic sonar image construction device is electrically connected to the plurality of sonar devices and is used to: construct a panoramic sonar image according to the sonar signal; and perform pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target in the actual space and the size of the target; The control device is electrically connected to the panoramic sonar image construction device and the deep-sea operation equipment, respectively, and is used to: determine the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; and, according to the current safety level of the deep-sea operation equipment, control the deep-sea operation equipment to perform corresponding operating actions.

[0025] It can be understood that the deep-sea operation equipment safety monitoring system based on panoramic sonar images provided in an embodiment of the present invention can execute all the steps and functions of the deep-sea operation equipment safety monitoring method based on panoramic sonar images provided in any embodiment of the present invention.

[0026] Specifically, a deep-sea operation equipment safety monitoring method based on panoramic sonar images in an embodiment of the present invention provides a deep-sea operation equipment safety early warning mechanism. Figure 2 As shown, a sonar device is installed in each of the four directions of the deep-sea operation equipment. After obtaining the sonar signal returned by the sonar device after detecting the seabed at various angles, each sonar device transmits its sonar signal to the panoramic sonar image construction device to construct a 360° panoramic sonar image. In one embodiment, the panoramic sonar image can also be displayed in real time on the display screen. Secondly, the panoramic sonar image construction device measures the pixels of the panoramic sonar screen, obtains the coordinate value of the sonar device distance detection target in the actual space and the size of the target itself, and feeds the detection signal back to the control device. The control device determines the safety level faced by the deep-sea operation equipment through the safety condition prediction operation, and sends the corresponding control signal according to the safety level, so that the deep-sea operation equipment performs corresponding operation actions, such as: immediate stop, turn and other operation actions, to realize the safety warning mechanism of the deep-sea operation equipment and reduce the operation risk of the deep-sea operation equipment.

[0027] In a specific embodiment, the deep-sea working equipment includes a trencher, and the control device includes a controller.

[0028] In step S100, during the seabed detection process, sonar devices in various directions acquire seabed information by emitting sound waves and receiving reflected signals. Common sonar devices such as multi-beam echo sounder system (MBES) use multiple sound beams to simultaneously detect the seabed at different angles and can generate high-resolution seabed topographic maps; side-scan sonar (SSS) is mainly used for large-area seabed coverage detection and can identify obstacles and man-made structures on the seabed. The sound waves emitted by these sonar devices will be reflected when encountering different objects and terrains on the seabed, and the reflected signals are received by the receivers of the sonar devices, thereby forming sonar signals containing seabed information.

[0029] In step S200, the sonar signals collected by each sonar device are transmitted to the panoramic sonar image construction device, which is configured with complex algorithms and processing models, such as geometric projection mapping model, space transformation model and image reconstruction model, to process the sonar signals.

[0030] Specifically, the geometric projection mapping model projects the sonar signal from the time-frequency domain to the spatial domain to determine the position distribution of the sonar signal in space. Then, the spatial transformation model performs multi-dimensional transformation on the spatial domain data after projection mapping to generate a multi-dimensional spatial transformation matrix to optimize the spatial distribution of the data and improve the image resolution and clarity. Finally, the image reconstruction model generates a single-frame sonar image based on the multi-dimensional spatial transformation matrix, and combines multiple single-frame images into a 360° panoramic sonar image through image stitching and fusion technology.

[0031] In a specific embodiment, the panoramic sonar image construction device can refer to the seabed detection sonar image single-frame reconstruction system in the invention patent CN118799434A "A method and system for single-frame reconstruction of sonar images for seabed detection", which can implement the step S200, and no further explanation is given for this step.

[0032] In a specific embodiment, the image stitching and fusion technology includes the following steps: Feature extraction and matching: Extract feature information such as feature points, edges or textures from adjacent single-frame sonar images, and use feature matching algorithms, such as grayscale-based correlation matching, feature point-based SIFT, SURF algorithm, etc., to find the corresponding relationship and overlapping area between adjacent single-frame sonar images.

[0033] Image registration: Based on the feature matching results, the transformation parameters such as translation, rotation and scaling between adjacent single-frame sonar images are calculated, and the adjacent single-frame sonar images are accurately registered to align them in space.

[0034] Image fusion: Image fusion algorithms such as weighted average method and median filter method are used to fuse the overlapping areas of adjacent single-frame sonar images after registration to eliminate stitching traces, achieve smooth image transition, and generate complete and continuous panoramic sonar images.

[0035] In a specific embodiment, the step S300 performs pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target in the actual space and the size of the target, including the following steps: S310, calibrating the pixel resolution of the panoramic sonar image to obtain a calibration ratio; S320, measuring the pixel position and the number of pixels occupied by the target object in the panoramic sonar image; S330, calculating the detected coordinate value of the target object in the actual space and the size of the target object according to the calibration ratio, the pixel position and the number of pixels.

[0036] In this embodiment, the coordinate value of the target in the actual space and the size of the target are calculated by using the ratio between the image pixels and the actual size (i.e., the calibration ratio), the pixel position and the number of pixels occupied by the target in the panoramic sonar image. Specifically, the steps are as follows: Pixel resolution calibration: First, the pixel resolution of the panoramic sonar image needs to be calibrated. This can be achieved by performing sonar detection on a standard object of known distance and size and measuring the number of pixels it occupies in the sonar image. For example, place a standard object of known length L meters within the sonar detection range, and measure its corresponding pixel length in the panoramic sonar image as P pixels. The ratio of pixels to actual length k=L / P can be obtained.

[0037] Target size measurement: After obtaining the calibration ratio, for the detected target in the panoramic sonar image, measure the number of pixels it occupies in different directions in the panoramic sonar image. For example, if the number of pixels in the horizontal direction is Px and the number of pixels in the vertical direction is Py, then the horizontal size of the target is Dx=k×Px, and the vertical size is Dy=k×Py.

[0038] Calculation of target object coordinates: Based on the number of pixels, pixel position and the above-mentioned proportional relationship of the target object in the panoramic sonar image, combined with the position and posture information of each sonar device, the coordinate value and size of the target object in the actual space are further calculated.

[0039] In a specific embodiment, the step S310 calibrates the pixel resolution of the panoramic sonar image to obtain a calibration ratio, including the following steps: S311, obtaining a sonar signal of a standard object of known distance and known size within the detection range of the sonar device; S312, constructing a panoramic sonar image for calibration according to the sonar signal; S313, measuring the pixel position and the number of pixels occupied by the standard object in the panoramic sonar image used for calibration; S314. Determine, according to the pixel position and the number of pixels occupied by the standard object in the panoramic sonar image used for calibration, the proportional relationship between the pixels of the panoramic sonar image used for calibration and the distance and the known size, respectively, to obtain a calibration ratio.

[0040] In this embodiment, the ratio between pixels and actual length is determined by imaging a standard object of known distance and size within the sonar detection range to obtain a calibration ratio, which is simple and effective. It can be understood that this calibration step can be performed before the deep-sea operation equipment is operated, and the calibration ratio can be stored in the panoramic sonar image construction device, and the data can be retrieved when needed.

[0041] In a specific embodiment, the step S400 determines the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object, and includes the following steps: S410, predicting the operation risk of the deep-sea operation equipment according to the coordinate value of the target object in the actual space, the size of the target object, and the operation status of the deep-sea operation equipment; S420. Determine the current safety level of the deep-sea operation equipment according to the operation risk of the deep-sea operation equipment and a preset safety threshold.

[0042] In this embodiment, the operating status of the deep-sea operation equipment includes stopping, moving forward, turning, etc. The operating risk of the deep-sea operation equipment is predicted based on the coordinate value of the target object in the actual space, the size of the target object, and the operating status of the deep-sea operation equipment, which can be converted into a corresponding scoring value, and then combined with the preset safety threshold, the current safety level of the deep-sea operation equipment is determined. For example, the safety level includes the first level operating risk, the second level operating risk, and the third level operating risk, and the operating risks of each level increase in sequence. It is determined which level of the preset safety threshold the operating risk of the deep-sea operation equipment reaches to determine the current safety level of the deep-sea operation equipment.

[0043] In a specific embodiment, the safety level includes level one operation risk, level two operation risk and level three operation risk. The step S500 controls the deep-sea operation equipment to perform corresponding operation actions according to the current safety level of the deep-sea operation equipment, including the following steps: S510, when it is determined that the current safety level of the deep-sea operation equipment is the first-level operation risk, controlling the deep-sea operation equipment to continue to perform the current operation; S520, when it is determined that the current safety level of the deep-sea operation equipment is the second-level operation risk, controlling the deep-sea operation equipment to perform an evasive action; S530: When it is determined that the current safety level of the deep-sea operation equipment is the third-level operation risk, control the deep-sea operation equipment to immediately stop operation.

[0044] In this embodiment, the control device triggers the deep-sea operation equipment to perform corresponding operation actions according to the safety level obtained. For example, when it is determined that the current safety level of the deep-sea operation equipment is the first-level operation risk, the control device controls the deep-sea operation equipment to continue to perform the current operation; when it is determined that the current safety level of the deep-sea operation equipment is the second-level operation risk, it indicates that the deep-sea operation equipment may have a certain collision risk, and the control device will control the deep-sea operation equipment to perform evasive actions such as steering, thereby avoiding the operation risk of the deep-sea operation equipment; when it is determined that the current safety level of the deep-sea operation equipment is the third-level operation risk, it indicates that the deep-sea operation equipment is about to face serious dangers such as collision, and the control device will immediately issue a command to make the deep-sea operation equipment execute the action of immediately stopping operation. A safety early warning mechanism for deep-sea operation equipment is implemented.

[0045] In a specific embodiment, the method further comprises the following steps: S600: Control a display device to display the panoramic sonar image.

[0046] In this example, see Figure 2 The display device is electrically connected to the panoramic sonar image construction device. After generating the panoramic sonar image, the panoramic sonar image construction device sends the panoramic sonar image to the display device, and the display device displays the panoramic sonar image in real time for the operator to monitor, so that the operator can intuitively understand the seabed conditions around the deep-sea operation equipment.

[0047] In summary, an embodiment of the present invention provides a method for safety monitoring of deep-sea operating equipment based on panoramic sonar images. By acquiring sonar signals returned by all sonar devices after seabed detection when the deep-sea operating equipment is running, a panoramic sonar image is constructed according to the sonar signals, pixel measurement is performed on the panoramic sonar image, and the coordinate value of the detected target object in the actual space and the size of the target object are obtained. According to the coordinate value of the target object in the actual space and the size of the target object, the current safety level of the deep-sea operating equipment is determined, and according to the current safety level of the deep-sea operating equipment, the deep-sea operating equipment is controlled to perform corresponding operating actions, thereby realizing a safety early warning mechanism for the deep-sea operating equipment, avoiding the operating risks of the deep-sea operating equipment, and increasing the service life of the deep-sea operating equipment.

[0048] See also Figure 3 , Figure 3 It is a structural schematic diagram of a deep-sea operation equipment safety monitoring device based on panoramic sonar images provided by one embodiment of the present invention.

[0049] In an embodiment of the present invention, a deep-sea operation equipment safety monitoring device based on panoramic sonar images is provided. Sonar equipment is installed in each direction of the deep-sea operation equipment. The device comprises: A sonar signal acquisition module, used to acquire the sonar signals returned by all the sonar devices after seabed detection when the deep-sea operation equipment is in operation; A panoramic sonar image construction module, used to construct a panoramic sonar image according to the sonar signal; A pixel measurement module is used to perform pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target in the actual space and the size of the target; A safety level determination module, used to determine the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; The operation action execution module is used to execute corresponding operation actions according to the current safety level of the deep-sea operation equipment.

[0050] An embodiment of the present invention provides a deep-sea operating equipment safety monitoring device based on panoramic sonar images, which can execute all the steps and functions of a deep-sea operating equipment safety monitoring method based on panoramic sonar images provided in any of the above embodiments, and the specific functions of the device are not described in detail here.

[0051] See also Figure 4 , Figure 4 It is a structural diagram of a terminal device provided by an embodiment of the present invention.

[0052] The terminal device includes: a processor, a memory, and a computer program stored in the memory and configured to be run by the processor. When the processor executes the computer program, the steps of a method for safety monitoring of deep-sea operation equipment based on panoramic sonar images in each of the above embodiments are implemented, for example: Figure 1 Alternatively, the processor implements the functions of each module in the above-mentioned device embodiments when executing the computer program.

[0053] Exemplarily, the computer program may be divided into one or more modules, and the one or more modules are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device. For example, the computer program may be divided into several modules, and the specific functions of each module have been described in detail in a deep-sea operating equipment safety monitoring method based on panoramic sonar images provided in any of the above embodiments, and the specific functions of the device will not be repeated here.

[0054] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art may understand that the schematic diagram is only an example of a terminal device and does not constitute a limitation on a terminal device. The terminal device may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0055] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect the various parts of the entire terminal device.

[0056] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the deep-sea operation equipment safety monitoring method based on panoramic sonar images by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0057] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a deep-sea operation equipment safety monitoring method based on panoramic sonar images as in any of the above embodiments.

[0058] If the module integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0059] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for safety monitoring of deep-sea operation equipment based on panoramic sonar images, characterized in that: Sonar equipment is installed in each direction of the deep-sea operation equipment, and the method includes: Acquiring sonar signals returned by all the sonar devices after seabed detection when the deep-sea operation equipment is in operation; constructing a panoramic sonar image according to the sonar signal; Performing pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target object in the actual space and the size of the target object; Determining the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; According to the current safety level of the deep-sea operation equipment, the deep-sea operation equipment is controlled to perform corresponding operation actions.

2. The method for safety monitoring of deep-sea operation equipment based on panoramic sonar images according to claim 1 is characterized in that: The pixel measurement of the panoramic sonar image to obtain the coordinate value of the detected target object in the actual space and the size of the target object includes: Calibrate the pixel resolution of the panoramic sonar image to obtain a calibration ratio; Measuring the pixel position and the number of pixels occupied by the target object in the panoramic sonar image; The detected coordinate value of the target object in the actual space and the size of the target object are calculated according to the calibration ratio, the pixel position and the pixel quantity.

3. The method for safety monitoring of deep-sea operation equipment based on panoramic sonar images according to claim 2 is characterized in that: The step of calibrating the pixel resolution of the panoramic sonar image to obtain a calibration ratio includes: Acquire a sonar signal of a standard object of known distance and known size within the detection range of the sonar device; constructing a panoramic sonar image for calibration according to the sonar signal; Measuring the pixel position and the number of pixels occupied by the standard object in the panoramic sonar image used for calibration; According to the pixel position and the number of pixels occupied by the standard object in the panoramic sonar image used for calibration, the proportional relationship between the pixels of the panoramic sonar image used for calibration and the distance and the known size is determined to obtain a calibration ratio.

4. The method for safety monitoring of deep-sea operation equipment based on panoramic sonar images according to claim 1 is characterized in that: Determining the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object includes: Predicting the operation risk of the deep-sea operation equipment according to the coordinate value of the target object in the actual space, the size of the target object, and the operation status of the deep-sea operation equipment; The current safety level of the deep-sea operation equipment is determined based on the operation risk of the deep-sea operation equipment and a preset safety threshold.

5. The method for safety monitoring of deep-sea operation equipment based on panoramic sonar images according to claim 1 is characterized in that: The safety level includes level one operation risk, level two operation risk and level three operation risk. According to the current safety level of the deep-sea operation equipment, the deep-sea operation equipment is controlled to perform corresponding operation actions, including: When it is determined that the current safety level of the deep-sea operation equipment is the first-level operation risk, controlling the deep-sea operation equipment to continue to perform the current operation; When it is determined that the current safety level of the deep-sea operation equipment is the second-level operation risk, controlling the deep-sea operation equipment to perform an evasive action; When it is determined that the current safety level of the deep-sea operation equipment is the third-level operation risk, the deep-sea operation equipment is controlled to immediately stop operation.

6. The method for safety monitoring of deep-sea operation equipment based on panoramic sonar images according to any one of claims 1 to 5, characterized in that: The method further comprises: Control a display device to display the panoramic sonar image.

7. A deep-sea operation equipment safety monitoring device based on panoramic sonar images, characterized in that: Sonar equipment is installed in each direction of the deep-sea operation equipment, and the device includes: A sonar signal acquisition module, used to acquire the sonar signals returned by all the sonar devices after seabed detection when the deep-sea operation equipment is in operation; A panoramic sonar image construction module, used to construct a panoramic sonar image according to the sonar signal; A pixel measurement module is used to perform pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target in the actual space and the size of the target; A safety level determination module, used to determine the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; The operation action execution module is used to execute corresponding operation actions according to the current safety level of the deep-sea operation equipment.

8. A deep-sea operation equipment safety monitoring system based on panoramic sonar images, characterized in that: Including multiple sonar equipment, deep-sea operation equipment, panoramic sonar image construction equipment and control equipment; A plurality of the sonar devices are respectively installed in various directions of the deep-sea operation equipment, and are used to obtain sonar signals returned after seabed detection when the deep-sea operation equipment is in operation; The panoramic sonar image construction device is electrically connected to the plurality of sonar devices and is used to: construct a panoramic sonar image according to the sonar signal; and perform pixel measurement on the panoramic sonar image to obtain the coordinate value of the detected target in the actual space and the size of the target; The control device is electrically connected to the panoramic sonar image construction device and the deep-sea operation equipment, respectively, and is used to: determine the current safety level of the deep-sea operation equipment according to the coordinate value of the target object in the actual space and the size of the target object; and, according to the current safety level of the deep-sea operation equipment, control the deep-sea operation equipment to perform corresponding operating actions.

9. A terminal device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for safety monitoring of deep-sea operation equipment based on panoramic sonar images as described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a deep-sea operation equipment safety monitoring method based on panoramic sonar images as described in any one of claims 1 to 6.

Images