Method and device for monitoring equipment in dredger cabin, server and storage medium

Abstract

The invention discloses an equipment monitoring method and device for a dredger cabin, a server and a storage medium, and relates to the technical field of dredging, the method comprises the steps that visual images and infrared images of the dredger cabin are acquired, and the visual images comprise global visual images and local visual images; inputting the visual image and the infrared image into a pre-trained target detection model, so that the target detection model determines a target detection result according to the visual image and the infrared image; and when it is determined that the target detection result comprises at least one abnormal target, it is determined that equipment in the dredger cabin is abnormal, and the abnormal targets at least comprise abnormal flames, abnormal smoke, abnormal oil and abnormal personnel. According to the technical scheme, real-time target detection of the dredger cabin is achieved, then real-time monitoring of the operation state of the equipment in the dredger cabin is achieved, workers can conveniently and timely maintain and process the abnormal state of the equipment in the dredger cabin, and the operation safety of the dredger is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of dredging technology, and in particular to a method, device, server, and storage medium for monitoring indoor equipment in a dredging vessel cabin. Background Art

[0002] In recent years, with the growing demand for global infrastructure construction, dredgers have played a vital role in key areas such as roadway dredging and port construction. However, during dredger operation, safety hazards such as high-temperature fires, equipment failures, and improper operation by personnel are constantly emerging, posing severe challenges to personnel safety, equipment integrity, and environmental safety.

[0003] In the prior art, supervision of equipment in dredging cabins mainly relies on personnel inspections, and the operating status of the equipment in dredging cabins is monitored by arranging multiple frequency inspections.

[0004] However, this method is inefficient and has a low accuracy rate in judging the operating status. Summary of the Invention

[0005] The present invention provides a method, device, server and storage medium for monitoring equipment in a dredging vessel's cabin. By acquiring and analyzing real-time images of the dredging vessel, real-time monitoring of the equipment in the dredging vessel's cabin is achieved, thereby improving the accuracy of judging the operating status of the equipment in the cabin and thereby improving the safety of the dredging vessel's operation.

[0006] In a first aspect, an embodiment of the present invention provides a method for monitoring equipment in a dredging vessel cabin, comprising:

[0007] Acquire a visual image and an infrared image of the dredger cabin, wherein the visual image includes a global visual image and a local visual image;

[0008] Inputting the visual image and the infrared image into a pre-trained target detection model so that the target detection model determines a target detection result based on the visual image and the infrared image;

[0009] When it is determined that the target detection result includes at least one abnormal target, it is determined that the equipment in the dredging cabin is abnormal, wherein the abnormal target includes at least abnormal flame, abnormal smoke, abnormal oil and abnormal personnel.

[0010] The technical solution of an embodiment of the present invention provides a method for monitoring equipment in a dredging vessel cabin, comprising: acquiring a visual image and an infrared image of the dredging vessel cabin, wherein the visual image includes a global visual image and a local visual image; inputting the visual image and the infrared image into a pre-trained target detection model so that the target detection model determines a target detection result based on the visual image and the infrared image; and determining that the equipment in the dredging vessel cabin is abnormal when it is determined that the target detection result includes at least one abnormal target, wherein the abnormal target includes at least abnormal flames, abnormal smoke, abnormal oil, and abnormal personnel. The above technical solution can first obtain visual images and infrared images of the dredger cabin, realize visual monitoring and temperature monitoring of the dredger cabin, and the global visual image and local visual image contained in the visual image can realize overall visual monitoring of the dredger cabin, thereby improving the comprehensiveness of visual monitoring. Secondly, the visual image and infrared image can be input into the target detection model so that the target detection model determines the target detection result based on the visual image and infrared image. The target detection model performs real-time analysis of the visual image and infrared image of the dredger cabin, thereby realizing real-time target detection of the dredger cabin. When it is determined that the detection targets contained in the target detection result include at least one abnormal target, it is determined that the equipment in the dredger cabin is abnormal. According to whether the target detection result contains the abnormal target, the operating status of the equipment in the dredger cabin is monitored in real time, thereby improving the accuracy of the operating status judgment. In addition, it is convenient for the staff to carry out timely maintenance and processing of the abnormal status of the equipment in the dredger cabin, thereby improving the safety of the dredger operation.

[0011] Furthermore, after obtaining the visual image and infrared image of the dredger cabin, the method further includes:

[0012] The visual image is optimized by performing image enhancement, image denoising and image scaling on the visual image to obtain an optimized visual image.

[0013] Furthermore, after obtaining the visual image and infrared image of the dredger cabin, the method further includes:

[0014] determining a device area in the partial visual image, and determining a target area in the partial visual image according to the device area;

[0015] Deleting other areas except the target area in the local visual image to obtain an optimized local visual image.

[0016] Furthermore, the target detection result includes the detected target and status information of the detected target, and the status information includes category information and location information of the detected target.

[0017] Furthermore, after the target detection model determines the target detection result according to the visual image and the infrared image, the method further includes:

[0018] If the category information of the detection target is a person and the degree of overlap between the person's position information and the device position information of the marking device is greater than a preset threshold, the detection target is determined to be an abnormal person.

[0019] Furthermore, after determining that the equipment in the dredging cabin is abnormal, the method further includes:

[0020] Determine warning information according to the abnormal target and the status information of each abnormal target, and send the warning information to a monitoring terminal.

[0021] Furthermore, the training process of the target detection model includes:

[0022] Acquire multiple historical visual images of the dredger cabin and historical infrared images corresponding to each historical visual image;

[0023] Using each of the historical visual images and the corresponding historical infrared images as training images, determining detection targets in each of the training images and status information of each detection target;

[0024] The training image is used as a training input, and the detection targets in the training image and the status information of each detection target are used as a training output to perform network training to obtain the target detection model.

[0025] In a second aspect, an embodiment of the present invention further provides a device for monitoring equipment in a dredging cabin, comprising:

[0026] an acquisition module, configured to acquire a visual image and an infrared image of the dredger cabin, wherein the visual image includes a global visual image and a local visual image;

[0027] a determination module, configured to input the visual image and the infrared image into a pre-trained target detection model, so that the target detection model determines a target detection result based on the visual image and the infrared image;

[0028] The execution module is used to determine that the indoor equipment of the dredging vessel cabin is abnormal when it is determined that the target detection result includes at least one abnormal target, wherein the abnormal target at least includes abnormal flames, abnormal smoke, abnormal oil and abnormal personnel.

[0029] In a third aspect, an embodiment of the present invention further provides a server, comprising:

[0030] at least one processor; and a memory communicatively coupled to the at least one processor;

[0031] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for monitoring indoor equipment in a dredging vessel cabin as described in any one of the first aspects.

[0032] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to execute the method for monitoring indoor equipment in a dredging vessel cabin as described in any one of the first aspects.

[0033] In a fifth aspect, the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed on a computer, the computer executes the method for monitoring indoor equipment in a dredging vessel cabin as provided in the first aspect.

[0034] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the dredging cabin indoor equipment monitoring device, or may be packaged separately from the processor of the dredging cabin indoor equipment monitoring device, and this application does not limit this.

[0035] The descriptions of the second, third, fourth and fifth aspects of this application can refer to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth and fifth aspects can refer to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0036] In this application, the name of the aforementioned dredging vessel cabin equipment monitoring device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they are within the scope of the claims of this application and their equivalents.

[0037] These and other aspects of the present application will become more readily apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A flowchart of a method for monitoring equipment in a dredging vessel cabin provided by an embodiment of the present invention;

[0040] Figure 2 A schematic structural diagram of a dredger monitoring system provided by an embodiment of the present invention;

[0041] Figure 3 A flow chart of another method for monitoring equipment in a dredging vessel cabin provided by an embodiment of the present invention;

[0042] Figure 4 A schematic structural diagram of a dredging cabin indoor equipment monitoring device provided by an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of the structure of a server provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0045] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0046] The terms "first" and "second" and the like in the specification and drawings of this application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0047] Furthermore, the terms "including," "having," and any variations thereof, as used in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0048] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. When its operation is completed, the process can be terminated, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the features in the embodiments of the present invention and the embodiments can be combined with each other without conflict.

[0049] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being more preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0050] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0051] Image recognition technology, a key achievement in information technology, has achieved remarkable results in various fields. To improve the efficiency of equipment inspections and the accuracy of operational status assessments within dredging vessels, image recognition technology can be used to monitor the status of these equipment. However, the complex structure of dredging vessel equipment makes the application of image recognition technology challenging.

[0052] Therefore, the present application proposes a method for monitoring equipment in a dredging cabin, which can realize real-time monitoring of the equipment in the dredging cabin and improve the safety of the dredging operation.

[0053] The method for monitoring indoor equipment in a dredging vessel cabin proposed in this application will be described in detail below with reference to diagrams and embodiments.

[0054] Figure 1 This is a flow chart of a method for monitoring indoor equipment in a dredging cabin provided by an embodiment of the present invention. This embodiment is applicable to situations where it is necessary to monitor the operation of indoor equipment in a dredging cabin. The method can be executed by a monitoring device for indoor equipment in a dredging cabin. The monitoring device for indoor equipment in a dredging cabin can be configured in a dredging ship monitoring system. Figure 2 A schematic diagram of the structure of a dredger monitoring system provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the dredger monitoring system includes an image acquisition module, a server equipped with a dredger cabin indoor equipment monitoring device, and a monitoring terminal. Figure 1As shown, the specific steps include:

[0055] Step 110: Acquire visual images and infrared images of the dredger cabin.

[0056] The visual image includes a global visual image and a local visual image.

[0057] A dredger can be a trailing suction hopper dredger, which includes cabins such as the engine room, rudder room, pump room, frequency conversion room, and hydraulic pump room. The equipment in these cabins has a special installation environment and complex wiring connections. Specifically, in order to perform overall visual monitoring of the dredger's cabins, visual images can be acquired based on the image acquisition device included in the image acquisition module. The image acquisition device includes a first image acquisition device installed at a reasonable position and angle in each cabin of the dredger and a second image acquisition device installed at a reasonable position and angle around each device in the dredger's cabin. Therefore, a global visual image of each cabin of the dredger can be acquired based on the first image acquisition device, and a local visual image of each device in each cabin can be acquired based on the second image acquisition device, thereby achieving visual monitoring of the dredger's cabins and ensuring that the picture is clear, unobstructed, and that the image can be captured completely.

[0058] During normal operation of a dredger, the temperature of the equipment in each cabin remains below a temperature threshold. If the temperature of any device exceeds the threshold, it indicates that the device may be malfunctioning, and thus, the dredger's operating status may be abnormal. Therefore, real-time monitoring of the temperature distribution in the dredger's cabin is necessary. The temperature distribution of each device in the dredger's cabin can be reflected through infrared images, and the cabin temperature can be monitored in real time by acquiring infrared images of the dredger's cabin. Specifically, the image acquisition module includes an infrared image acquisition device positioned at a suitable position and angle within the dredger's cabin to acquire a global infrared image of the dredger's cabin, thereby enabling temperature monitoring of the dredger's cabin.

[0059] In the embodiment of the present invention, visual monitoring and temperature monitoring of the dredger cabin are achieved by acquiring visual images and infrared images of the dredger cabin.

[0060] Step 120: Input the visual image and the infrared image into a pre-trained target detection model, so that the target detection model determines a target detection result based on the visual image and the infrared image.

[0061] Among them, the target detection model has a built-in multimodal target detection algorithm. The multimodal target detection algorithm determines the detection target and its status information by analyzing the visual image and infrared image of the dredger cabin. The status information of the detection target can include category information and location information. The detection target can be represented by a region of interest (ROI) box, and the location information can be represented by the center coordinates, length and width.

[0062] Moreover, the target detection model is obtained by training a network model based on historical visual images and historical infrared images of the dredger cabin and corresponding target detection results, and has the ability to output target detection results based on the input visual images and infrared images of the dredger cabin.

[0063] Specifically, the visual and infrared images of the dredger's cabin are input into the target detection model. The target detection model analyzes the visual and infrared images to determine the detected target and its status information. The target category information here can include devices and personnel. It is understood that if a high-temperature fire occurs in the dredger's cabin, the target category information can also include flames and smoke. If an oil leak occurs in the dredger's cabin, the target category information can also include oil. If a person intrudes abnormally in the dredger's cabin, the person is considered an abnormal person.

[0064] It should be noted that abnormal intrusion can be understood as a person approaching a marking device that is not allowed to approach, and an abnormal person can be understood as a person approaching the marking device. Whether a person is an abnormal person can be determined by the overlap between the person and the marking device. Specifically, the overlap between the person's ROI box and the marking device's ROI box can be determined. If the overlap exceeds a preset threshold, the person is determined to be an abnormal person.

[0065] In addition, the degree of overlap between the two ROI frames may be determined based on the position information of each ROI frame, that is, the degree of overlap between the two ROI frames may be determined based on the center coordinates, length, and width of each ROI frame.

[0066] In an embodiment of the present invention, by inputting the visual image and infrared image of the dredger cabin into the target detection model, the target detection model determines the target detection result based on the visual image and infrared image of the dredger cabin, thereby realizing real-time target detection of the dredger cabin and further realizing real-time monitoring of the dredger.

[0067] Step 130: When it is determined that the target detection result includes at least one abnormal target, determine that the equipment in the dredging cabin is abnormal.

[0068] The abnormal targets include at least abnormal flames, abnormal smoke, abnormal oil and abnormal personnel.

[0069] If the target detection result includes abnormal flames or abnormal smoke, it indicates a high-temperature fire in the dredger's cabin. If the target detection result includes abnormal oil, it indicates an oil leak in the dredger's cabin. If the target detection result includes abnormal personnel, it indicates an abnormal intrusion by a person in the dredger's cabin. Of course, if at least one of the following occurs in the dredger's cabin: a high-temperature fire, an oil leak, or an abnormal intrusion by a person, it can be determined that there is an abnormality in the dredger's cabin equipment.

[0070] Specifically, after obtaining the target detection results output by the target detection model, the target detection results can be analyzed. Specifically, the detection targets contained in the target detection results can be analyzed. If the detection targets include at least one abnormal target, that is, if the detection targets include at least one of abnormal flames, abnormal smoke, abnormal oil, and abnormal personnel, it can be determined that at least one of a high-temperature fire, an oil leak, and abnormal personnel intrusion has occurred in the dredger cabin, and therefore, it can be determined that the equipment in the dredger cabin is abnormal. Of course, if the detection targets do not include any of the abnormal flames, abnormal smoke, abnormal oil, and abnormal personnel, it can be determined that none of the high-temperature fire, oil leak, and abnormal personnel intrusion has occurred in the dredger cabin, and therefore, it can be determined that the equipment in the dredger cabin is normal.

[0071] In an embodiment of the present invention, by analyzing the target detection results, the status of the equipment in the dredging cabin is determined, and then the real-time monitoring of the equipment in the dredging cabin is achieved, which makes it convenient for the staff to perform timely maintenance and processing on the abnormal status of the equipment in the dredging cabin, thereby improving the safety of the dredging ship operation.

[0072] The method for monitoring indoor equipment in a dredging vessel cabin provided by an embodiment of the present invention includes: obtaining a visual image and an infrared image of the dredging vessel cabin, wherein the visual image includes a global visual image and a local visual image; inputting the visual image and the infrared image into a pre-trained target detection model so that the target detection model determines a target detection result based on the visual image and the infrared image; when it is determined that the target detection result includes at least one abnormal target, determining that the indoor equipment in the dredging vessel cabin is abnormal, wherein the abnormal target includes at least abnormal flames, abnormal smoke, abnormal oil and abnormal personnel. The above technical solution can first obtain visual images and infrared images of the dredger cabin, realize visual monitoring and temperature monitoring of the dredger cabin, and the global visual image and local visual image contained in the visual image can realize overall visual monitoring of the dredger cabin, thereby improving the comprehensiveness of visual monitoring. Secondly, the visual image and infrared image can be input into the target detection model so that the target detection model determines the target detection result based on the visual image and infrared image. The target detection model performs real-time analysis of the visual image and infrared image of the dredger cabin, thereby realizing real-time target detection of the dredger cabin. When it is determined that the detection targets contained in the target detection result include at least one abnormal target, it is determined that the equipment in the dredger cabin is abnormal. According to whether the target detection result contains the abnormal target, the operating status of the equipment in the dredger cabin is monitored in real time, thereby improving the accuracy of the operating status judgment. In addition, it is convenient for the staff to carry out timely maintenance and processing of the abnormal status of the equipment in the dredger cabin, thereby improving the safety of the dredger operation.

[0073] Figure 3 This is a flow chart of another method for monitoring equipment in a dredging vessel cabin provided by an embodiment of the present invention. This embodiment is specific based on the above embodiment. Figure 3 As shown, in this embodiment, the method may further include:

[0074] Step 310: Acquire visual images and infrared images of the dredging vessel cabin.

[0075] The visual image includes a global visual image and a local visual image.

[0076] Specifically, to enhance the overall visual monitoring of complex dredger cabins, a global visual image of each cabin can be acquired using a first image acquisition device installed in each cabin. Furthermore, a local visual image of each device inside the cabin can be acquired using a second image acquisition device installed at appropriate locations and angles around the device. To monitor the temperature of the dredger cabin, a global infrared image of the dredger can be acquired using infrared image acquisition devices installed at appropriate locations and angles within each cabin.

[0077] It should be noted that corresponding second image acquisition devices can be configured for each device in the dredging cabin, or a second image acquisition device can be configured for at least one device in the dredging cabin to acquire local visual images of each device.

[0078] In order to improve the integrity of dredger monitoring, first image acquisition devices and infrared image acquisition devices can be installed at reasonable positions and angles around the dredger to obtain global visual images and global infrared images of the dredger. Second image acquisition devices can also be installed at reasonable positions and angles around rake arms, mud pumps, generators, mud doors, bridges, electrical control boxes, shipborne system servers, and other non-cabin equipment with special installation environments to obtain local visual images of these equipment. Infrared image sensors can also be installed at reasonable positions and angles around equipment prone to high-temperature fires to obtain local infrared images of these equipment.

[0079] Of course, the target detection model can determine the target detection results based on the visual image and infrared image of the dredger cabin, the global visual image of the dredger and the local visual image of the aforementioned equipment, and the global infrared image of the dredger and the local infrared image of the equipment prone to high-temperature fire, so as to realize the operation monitoring of the dredger.

[0080] In one embodiment, after acquiring the visual image, the method further includes:

[0081] The visual image is optimized by performing image enhancement, image denoising and image scaling on the visual image to obtain an optimized visual image.

[0082] In order to improve the quality of the visual image captured by the image acquisition device and thereby improve the recognition effect of the target detection model, the visual image can be preprocessed. Specifically, the visual image can be optimized by performing image enhancement, image denoising, and image scaling on the visual image to obtain an optimized visual image. For example, the visual image can be enhanced using methods such as histogram equalization and contrast stretching, the visual image can be denoised using Gaussian filtering, and the visual image can be scaled to the size required by the target detection model as input information to ensure that the target detection model can smoothly process the visual image.

[0083] The visual image includes a local visual image and a global visual image. The local visual image may include a specific area device. In order to avoid the influence of the image periphery (other devices outside the specific area device) on the target detection result, the local visual image can be segmented.

[0084] In another embodiment, after acquiring the visual image, the method further includes:

[0085] Determine a device area in the partial visual image, determine a target area in the partial visual image according to the device area; and delete other areas except the target area in the partial visual image to obtain an optimized partial visual image.

[0086] For the local visual images acquired by the second image acquisition device, device regions can be pre-set for these local visual images. The device region in the local visual image can be understood as the image of the specific area device monitored by the image acquisition device corresponding to the local visual image.

[0087] After acquiring the partial visual image, the device area corresponding to the partial visual image can be determined first. Secondly, the target area can be determined in the partial visual image based on the device area corresponding to the partial visual image. Specifically, the partial visual image and the device area corresponding to the partial visual image can be processed based on a template matching algorithm, and the area in the partial visual image that is most similar to the device area can be determined as the target area. Then, other areas outside the target area in the partial visual image can be deleted. Specifically, other areas can be deleted based on the region of interest selection method, thereby deleting the image periphery in the partial visual image, thereby optimizing the partial visual image and obtaining an optimized partial visual image.

[0088] In an embodiment of the present invention, visual monitoring and temperature monitoring of the dredger cabin are achieved by acquiring visual images and infrared images of the dredger cabin. By preprocessing and optimizing the acquired visual images, the quality of the visual images is improved, making the visual images more compatible with the target detection model.

[0089] Step 320: Input the visual image and the infrared image into a pre-trained target detection model, so that the target detection model determines a target detection result based on the visual image and the infrared image.

[0090] The target detection result includes the detected target and status information of the detected target, and the status information includes category information and location information of the detected target.

[0091] As mentioned above, the visual image and infrared image of the dredger cabin are input into the target detection model. The target detection model can determine the target detection result by analyzing the visual image and the infrared image, and specifically determine the detection target as well as the category information and location information of the detection target.

[0092] If the category information of the detection target is a person and the overlap between the ROI box of the person and the ROI box of the marking device is greater than a preset threshold, it can be determined that the detection target is an abnormal person.

[0093] When the dredger is in a normal state, the category information of the detection target includes equipment and personnel, where the personnel are normal personnel; when the dredger is in an abnormal state, the category information of the detection target includes at least one of flame, smoke, oil and abnormal personnel.

[0094] In one embodiment, the training process of the target detection model includes:

[0095] Acquire multiple historical visual images of the dredger cabin and historical infrared images corresponding to each historical visual image; use each of the historical visual images and the corresponding historical infrared image as a training image, and determine the detection targets in each training image and the status information of each detection target; use the training images as training inputs and the detection targets in the training images and the status information of each detection target as training outputs to perform network training to obtain the target detection model.

[0096] In the specific implementation, multiple historical visual images of the dredger cabin and historical infrared images corresponding to each historical visual image can be obtained, and the detection targets in each historical visual image and the corresponding historical infrared image as well as the category information and position information of each detection target can be determined. Each historical visual image and the corresponding historical infrared image are used as training input, and the detection targets in each historical visual image and the corresponding historical infrared image as well as the category information and position information of each detection target are used to guide the training output for network training. During the training process, the model parameters are reversely optimized through the loss function, and the model is evaluated through cross-validation, accuracy, recall rate, F1 score (harmonic mean of accuracy and recall rate) and other indicators until the model indicators meet the corresponding indicator thresholds, thereby obtaining the target detection model.

[0097] In an embodiment of the present invention, by inputting the visual image and infrared image of the dredger cabin into the target detection model, the target detection model determines the target detection result based on the visual image and infrared image of the dredger, thereby realizing real-time target detection of the dredger cabin and further realizing real-time monitoring of the dredger cabin.

[0098] Step 330: When it is determined that the target detection result includes at least one abnormal target, determine that the equipment in the dredging cabin is abnormal.

[0099] The abnormal targets include at least abnormal flames, abnormal smoke, abnormal oil and abnormal personnel.

[0100] As previously described, after the target detection model outputs the target detection results, the target detection results can be analyzed. If the target detection results contain at least one of the following categories of detection targets: abnormal flames, abnormal smoke, abnormal oil, and abnormal personnel, it can be determined that at least one of a high-temperature fire, an oil leak, and abnormal personnel intrusion has occurred in the dredger's cabin, and therefore, it can be determined that the equipment in the dredger's cabin is abnormal. Of course, if the target detection results contain no category information of the detection targets: abnormal flames, abnormal smoke, abnormal oil, and abnormal personnel, it can be determined that none of the following categories of detection targets have occurred in the dredger, and therefore, it can be determined that the equipment in the dredger's cabin is normal.

[0101] In an embodiment of the present invention, by analyzing the target detection results, the status of the equipment in the dredging cabin is determined, and then the real-time monitoring of the status of the equipment in the dredging cabin is achieved, which makes it convenient for the staff to perform timely maintenance and processing on the abnormal status of the equipment in the dredging cabin, thereby improving the safety of the dredging ship operation.

[0102] Step 340: Determine warning information based on the abnormal target and the status information of each abnormal target, and send the warning information to a monitoring terminal.

[0103] Specifically, after determining that the status of the equipment in the dredging cabin is abnormal, the abnormal targets that cause the abnormal status of the dredger and the category information and location information of each abnormal target can be sent to the monitoring terminal as early warning information, so that the staff can remotely check the abnormal operating status of the equipment in the dredger cabin through the monitoring terminal in time.

[0104] In addition, the acquired visual images and infrared images of the dredger cabin can be sent to the monitoring terminal in real time, so that the staff can view the operation status of the dredger cabin in real time, providing intuitive and comprehensive dredger monitoring services.

[0105] In the embodiment of the present invention, by sending abnormal targets and status information of abnormal targets as early warning information to the monitoring terminal, timely prompting and display of abnormal status of indoor equipment in the dredging cabin can be achieved.

[0106] The method for monitoring indoor equipment in a dredging vessel cabin provided by an embodiment of the present invention includes: obtaining a visual image and an infrared image of the dredging vessel cabin; inputting the visual image and the infrared image into a pre-trained target detection model so that the target detection model determines a target detection result based on the visual image and the infrared image; when it is determined that the target detection result includes at least one abnormal target, determining that the indoor equipment in the dredging vessel cabin is abnormal; determining early warning information based on the abnormal target and the status information of each abnormal target, and sending the early warning information to a monitoring terminal. The above technical solution can first obtain the global visual image, local visual image and infrared image of the dredger cabin, realize the overall visual monitoring and temperature monitoring of the dredger cabin, and improve the comprehensiveness of the monitoring. Secondly, the visual image and the infrared image can be input into the target detection model, so that the target detection model determines the target detection result based on the visual image and the infrared image, that is, determines the detection target and the category information and position information of the detection target. Through the real-time analysis of the visual image and the infrared image of the dredger cabin by the target detection model, continuous and stable real-time target detection of the dredger cabin can be realized. When it is determined that the detection targets contained in the target detection result include at least one abnormal target, the dredger cabin is determined. Indoor equipment abnormality: based on whether the category information of the detection target contained in the target detection result contains abnormal targets, automatic real-time monitoring of the operating status of the indoor equipment in the dredger cabin is realized, thereby improving the efficiency, depth, breadth and accuracy of monitoring and reducing the monitoring cost. When it is determined that the category information of the detection target contained in the target detection result contains abnormal targets, the operation of the indoor equipment in the dredger cabin is determined to be abnormal, and the abnormal targets and the category information and location information of the abnormal targets are sent to the monitoring terminal as early warning information, thereby realizing early warning of the abnormal status of the indoor equipment in the dredger cabin, making it convenient for the staff to make corresponding treatment of the dredger according to the early warning information, timely eliminate safety hazards, and improve the safety of the dredger operation.

[0107] Figure 4 This is a schematic diagram of a dredging vessel cabin equipment monitoring device, provided in an embodiment of the present invention. This device is suitable for use in situations where it is necessary to monitor the operation of dredging vessel cabin equipment. This device can be implemented using software and / or hardware and is typically integrated into electronic equipment, such as a dredging vessel monitoring system.

[0108] like Figure 4 As shown, the device includes:

[0109] An acquisition module 410 is configured to acquire a visual image and an infrared image of the dredger cabin, wherein the visual image includes a global visual image and a local visual image;

[0110] a determination module 420 for inputting the visual image and the infrared image into a pre-trained target detection model, so that the target detection model determines a target detection result based on the visual image and the infrared image;

[0111] The execution module 430 is configured to determine that the equipment in the dredging vessel cabin is abnormal when it is determined that the target detection result includes at least one abnormal target, wherein the abnormal target includes at least abnormal flame, abnormal smoke, abnormal oil and abnormal personnel.

[0112] The dredging vessel cabin interior equipment monitoring device provided in this embodiment obtains visual images and infrared images of the dredging vessel cabin, wherein the visual images include global visual images and local visual images; inputs the visual images and the infrared images into a pre-trained target detection model so that the target detection model determines a target detection result based on the visual images and the infrared images; and when it is determined that the target detection result includes at least one abnormal target, determines that the dredging vessel cabin interior equipment is abnormal, wherein the abnormal targets include at least abnormal flames, abnormal smoke, abnormal oil, and abnormal personnel. The above technical solution can first obtain visual images and infrared images of the dredger cabin, realize visual monitoring and temperature monitoring of the dredger cabin, and the global visual image and local visual image contained in the visual image can realize overall visual monitoring of the dredger cabin, thereby improving the comprehensiveness of visual monitoring. Secondly, the visual image and infrared image can be input into the target detection model so that the target detection model determines the target detection result based on the visual image and infrared image. The target detection model performs real-time analysis of the visual image and infrared image of the dredger cabin, thereby realizing real-time target detection of the dredger cabin. When it is determined that the detection targets contained in the target detection result include at least one abnormal target, it is determined that the equipment in the dredger cabin is abnormal. According to whether the target detection result contains the abnormal target, the operating status of the equipment in the dredger cabin is monitored in real time, thereby improving the accuracy of the operating status judgment. In addition, it is convenient for the staff to carry out timely maintenance and processing of the abnormal status of the equipment in the dredger cabin, thereby improving the safety of the dredger operation.

[0113] Based on the above embodiment, the device further includes:

[0114] The optimization module is used to optimize the visual image by performing image enhancement, image denoising and image scaling on the visual image after acquiring the visual image and infrared image of the dredger cabin to obtain an optimized visual image; it is also used to determine the equipment area in the local visual image after acquiring the visual image and infrared image of the dredger cabin, determine the target area in the local visual image based on the equipment area; and delete other areas other than the target area in the local visual image to obtain an optimized local visual image.

[0115] In one embodiment, the target detection result includes a detected target and status information of the detected target, and the status information includes category information and location information of the detected target.

[0116] Based on the above embodiment, the determination module 420 is further configured to:

[0117] After the target detection model determines the target detection result based on the visual image and the infrared image, if the category information of the detection target is a person and the overlap between the person position information and the device position information of the marking device is greater than a preset threshold, the detection target is determined to be an abnormal person.

[0118] Based on the above embodiment, the device further includes:

[0119] The early warning module is used to determine early warning information according to the abnormal target and the status information of each abnormal target after determining that the equipment in the dredging cabin is abnormal, and send the early warning information to the monitoring terminal.

[0120] Based on the above embodiment, the device further includes:

[0121] The training module is used to execute the training process of the target detection model, specifically for: obtaining multiple historical visual images of the dredger cabin and historical infrared images corresponding to each historical visual image; using each of the historical visual images and the corresponding historical infrared image as a training image, determining the detection targets in each of the training images and the status information of each detection target; using the training images as training inputs and the detection targets in the training images and the status information of each detection target as training outputs to perform network training to obtain the target detection model.

[0122] The dredging cabin indoor equipment monitoring device provided by the embodiment of the present invention can execute the dredging cabin indoor equipment monitoring method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the dredging cabin indoor equipment monitoring method.

[0123] It is worth noting that in the above-mentioned embodiment of the dredging cabin indoor equipment monitoring device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0124] Figure 5 A schematic diagram of the structure of a server provided in an embodiment of the present invention. Figure 5 A block diagram of an exemplary server 5 suitable for implementing embodiments of the present invention is shown. Figure 5The server 5 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0125] like Figure 5 As shown, the server 5 is in the form of a general-purpose computing electronic device. Components of the server 5 may include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 connecting various system components (including the system memory 28 and the processing unit 16).

[0126] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0127] The server 5 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the server 5, including volatile and non-volatile media, removable and non-removable media.

[0128] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Server 5 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 5 Not shown, often called a "hard drive"). Although Figure 5 Not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0129] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0130] The server 5 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the server 5, and / or any device that enables the server 5 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the server 5 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. Figure 5 As shown, the network adapter 20 communicates with other modules of the server 5 via the bus 18. Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the server 5, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0131] The processing unit 16 executes various functional applications and page displays by running programs stored in the system memory 28, for example, implementing the method for monitoring equipment in a dredging vessel cabin provided by an embodiment of the present invention, which includes:

[0132] Acquire a visual image and an infrared image of the dredger cabin, wherein the visual image includes a global visual image and a local visual image;

[0133] Inputting the visual image and the infrared image into a pre-trained target detection model so that the target detection model determines a target detection result based on the visual image and the infrared image;

[0134] When it is determined that the target detection result includes at least one abnormal target, it is determined that the equipment in the dredging cabin is abnormal, wherein the abnormal target includes at least abnormal flame, abnormal smoke, abnormal oil and abnormal personnel.

[0135] Of course, those skilled in the art will appreciate that the processor may also implement the technical solution of the method for monitoring indoor equipment in a dredging vessel cabin provided by any embodiment of the present invention.

[0136] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for monitoring equipment in a dredging vessel cabin provided in an embodiment of the present invention is implemented. The method includes:

[0137] Acquire a visual image and an infrared image of the dredger cabin, wherein the visual image includes a global visual image and a local visual image;

[0138] Inputting the visual image and the infrared image into a pre-trained target detection model so that the target detection model determines a target detection result based on the visual image and the infrared image;

[0139] When it is determined that the target detection result includes at least one abnormal target, it is determined that the equipment in the dredging cabin is abnormal, wherein the abnormal target includes at least abnormal flame, abnormal smoke, abnormal oil and abnormal personnel.

[0140] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0141] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0142] Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0143] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0144] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computing device. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computer device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

[0145] In addition, the acquisition, storage, use, and processing of data in the technical solution of the present invention comply with the relevant provisions of national laws and regulations.

[0146] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for monitoring equipment in a dredging cabin, characterized in that: include: Acquire a visual image and an infrared image of the dredger cabin, wherein the visual image includes a global visual image and a local visual image; Inputting the visual image and the infrared image into a pre-trained target detection model so that the target detection model determines a target detection result based on the visual image and the infrared image; When it is determined that the target detection result includes at least one abnormal target, it is determined that the equipment in the dredging cabin is abnormal, wherein the abnormal target includes at least abnormal flame, abnormal smoke, abnormal oil and abnormal personnel.

2. The method for monitoring equipment in a dredging vessel cabin according to claim 1, characterized in that: After acquiring visual and infrared images of the dredger compartment, it also includes: The visual image is optimized by performing image enhancement, image denoising and image scaling on the visual image to obtain an optimized visual image.

3. The method for monitoring equipment in a dredging vessel cabin according to claim 1, characterized in that: After acquiring visual and infrared images of the dredger compartment, it also includes: determining a device area in the partial visual image, and determining a target area in the partial visual image according to the device area; Other areas except the target area in the local visual image are deleted to obtain an optimized local visual image.

4. The method for monitoring equipment in a dredging vessel cabin according to claim 1, characterized in that: The target detection result includes the detected target and status information of the detected target, and the status information includes category information and location information of the detected target.

5. The method for monitoring equipment in a dredging vessel cabin according to claim 4, characterized in that: After the target detection model determines the target detection result according to the visual image and the infrared image, the method further includes: If the category information of the detection target is a person and the degree of overlap between the person's position information and the device position information of the marking device is greater than a preset threshold, the detection target is determined to be an abnormal person.

6. The method for monitoring equipment in a dredging vessel cabin according to claim 5, characterized in that: After determining that the equipment in the dredging vessel cabin is abnormal, the method further includes: Determine warning information according to the abnormal target and the status information of each abnormal target, and send the warning information to a monitoring terminal.

7. The method for monitoring equipment in a dredging vessel cabin according to claim 1, characterized in that: The training process of the target detection model includes: Acquire multiple historical visual images of the dredger cabin and historical infrared images corresponding to each historical visual image; Using each of the historical visual images and the corresponding historical infrared images as training images, determining detection targets in each of the training images and status information of each detection target; The training image is used as a training input, and the detection targets in the training image and the status information of each detection target are used as a training output to perform network training to obtain the target detection model.

8. A dredging cabin indoor equipment monitoring device, characterized in that: include: an acquisition module, configured to acquire a visual image and an infrared image of the dredger cabin, wherein the visual image includes a global visual image and a local visual image; a determination module, configured to input the visual image and the infrared image into a pre-trained target detection model, so that the target detection model determines a target detection result based on the visual image and the infrared image; The execution module is used to determine that the indoor equipment of the dredging vessel cabin is abnormal when it is determined that the target detection result includes at least one abnormal target, wherein the abnormal target at least includes abnormal flames, abnormal smoke, abnormal oil and abnormal personnel.

9. A server, characterized in that: The server includes: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for monitoring equipment in a dredging vessel according to any one of claims 1 to 7.

10. A storage medium containing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the method for monitoring equipment in a dredging vessel according to any one of claims 1 to 7.

Images